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PURPOSE 


The  purpose  of  the  Defense  Systems  Management  Review  is 
to  disseminate  information  concerning  new  developments 
and  effective  actions  taken  relative  to  the  management  of 
defense  systems  programs  and  defense  systems  acquisition. 

The  Review  is  designed  as  a vehicle  to  transmit,  between 
persons  in  positions  of  leadership  and  responsibility  in  the 
program  management  and  systems  acquisition  communities, 
information  on  policies,  trends,  events  and  current  thinking 
affecting  the  practice  of  program  management  and  defense 
systems  acquisition.  The  publication  serves  as  a means  for 
providing  an  historical  record  of  significant  information 
associated  with  defense  systems  acquisition/management 
concepts  and  practices. 

The  Review  supports  the  assigned  mission  of  the  Defense 
Systems  Management  College,  and  serves  as  a medium  for 
continuing  the  education  and  professional  development  of 
persons  in  the  field. 
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DEFENSE  SYSTEMS 
MANAGEMENT  COLLEGE 


The  Review  is  an  essential  medium  for  communicating  new  ideas  or 
restating  of  old  ideas  in  the  field  of  acquisition  management.  I have 
been  impressed  with  the  quality  of  each  issue  and  intend  to  maintain 
that  reputation.  Thus  I solicit  comments  from  readers,  both  pro  and 
con,  so  I know  where  to  direct  our  efforts. 

I believe  it  appropriate  that  each  issue  have  a central  theme  so  that 
the  articles  contribute  to  a specific  acquisition  area,  rather  than  stand 
alone.  To  this  end,  Vol  I,  No.  5,  will  be  devoted  to  Test  and  Evaluation 
with  future  issues  covering  Production,  Development,  Configuration 
Management,  Software  Acquisition  and  Management,  Test  Equipment 
Management  and  other  important  areas  of  the  acquisition  process. 

Major  General  Jack  Albert,  my  predecessor,  made  an  outstanding  con- 
tribution to  the  acquisition  management  process  in  establishing  this 
Review  and  I look  forward  to  carrying  on  the  effort. 


R.  G.  FREEMAN  III 
Rear  Admiral,  USN 
Commandant 
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Can  Weapon  System 
Procurement  Be  Managed 

by 

Albert  J.  Kelley,  President 

Arthur  D.  Little  Program  Systems  Management  Company 


The  complexities  in  the  systems  acquisition  or  procurement  process  for  complex  weapon 
systems  causes  many  to  throw  up  their  hands  and  say  that  such  programs  cannot  be 
managed  efficiently-that  delays  and  overruns  will  always  be  a part  of  the  militarv-indus- 
trial  method  of  doing  business. 

Must  it  always  be  so?  Not  necessarily,  but  the  problem  is  so  huge  that  it  is  difficult  to  grasp 
in  normal  management  terms.  Offered  here  by  Mr.  KeUey  are  suggestions  that  if  adopted 
could  perhaps  alleviate  the  problem. 


THE  PROBLEM 

Weapon  system  procurement  problems 
have  been  well  publicized.  With  increas- 
ing frequency,  the  media  has  reported  sched- 
ule delays  and  cost  overruns  in  such  programs 
as  the  Trident  Missile  System,  the  F14 
Fighter,  and  Navy  Ship  Construction  Pro- 
grams. Weapon  system  procurement,  often 
called  systems  acquisition,  comprises  research, 
development,  test  and  evaluation,  production 
and  delivery  of  complex  technical  systems 
that  require  many  years  from  conception  to 
birth  and  many  more  years  to  eventual  ma- 
turity. 

Schedule  delays  and  cost  overruns  in  such 
programs  impact  military  effectiveness  and 
national  budgets  and  also  have  a subtle  but 
serious  effect  on  advanced  research  and  de- 
velopment for  Defense.  In  order  to  cover  cost 
overruns  on  major  programs  with  deep  com- 
mitments, smaller  advanced  research  and  de- 
velopment programs  are  often  reduced  or 
eliminated.  As  a result,  the  long  range  poten- 
tial of  our  Defense  posture  and  future  weapons 


systems  is  mortgaged  to  pay  for  the  present. 

Fortunately,  some  efforts  are  under  way 
and  other  initiatives  can  be  undertaken  to 
bring  weapon  system  procurement  under 
sound  management  control.  The  problem 
does  not  defy  description.  It  will  take  time, 
the  efforts  of  many  and,  above  all,  strong 
leadership. 

CAUSES- 

APPARENT  AND  REAL 

Many  causes  are  apparent  for  time  delays 
and  cost  increases  in  major  programs.  Among 
those  most  often  cited  are  technical  problems, 
unpredicted  inflation,  and  changing  require- 
ments. While  these  factors  do,  in  fact,  have  an 
impact  on  weapons  systems  they  are  often 
symptoms  rather  than  causes. 

Too  often,  probing  beneath  the  obvious 
symptoms  reveals  the  real  causes;  manage- 
ment, procurement  procedures,  and  their 
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handmaiden,  decisionmaking.  The  industrial 
contractor  usually  bears  the  brunt  for  failure 
to  make  schedules  and  budgets  but  the 
government  and  its  procedures  are  often 
equally  and  sometimes  more  at  fault.  The  low 
man  on  the  totem  pole  usually  gets  the  blame 
when  often  the  problem  starts  near  the  top. 

The  problems  with  weapon  systems  pro- 
curement and  management  often  begin  at 
the  very  initiation  of  the  project.  Insuffi- 
cient mission  analysis  coupled  with  incom- 
plete conceptual  and  technical  definition 
can  lead  to  poorly  defined  bid  requests 
wherein  the  client  agency  is  not  completely 
clear  what  it  wants,  when  it  can  get  it,  and 
how  much  it  will  cost.  Decisions  which 
should  have  been  made  before  the  contract 
was  signed,  are  made  eventually,  but  after  a 
large  work  force  is  on  the  payroll  and  deci- 
sion delays  cost  a lot  more  money. 

Often,  initial  project  cost  estimates  for 
completion  of  development  or  completion  of 
a production  run  do  not  truly  reflect  all  ele- 
ments and  are  usually  biased  heavily  on  the 
optimistic  side.  A major  cause  of  this  phe- 
nomenon is  the  gauntlet  of  over  2 years  from 
project  initiation  to  receipt  of  first  funding. 
In  this  period  there  are  many  hurdles  and  re- 
views, each  step  seeking  to  trim  costs.  At  each 
review  step,  the  project  supporters  face  the 
choice  of  staunchly  defending  soft  cost  esti- 
mates against  the  risk  of  having  the  embryo 
project  cancelled  completely  or  yielding  to 
pressures  at  each  step  to  reduce  further. 

In  this  same  time  period,  usually  additional 
requirements  are  being  added  to  the  weapons 
system  as  more  mission  roles  and  more  sophis- 
ticated technology  requirements  continue  to 
be  defined.  As  the  project  is  being  put  to- 
gether and  before  it  has  been  funded,  forces 
are  at  work  trying  to  reduce  the  cost  esti- 
mates while  at  the  same  time  adding  more 
capability,  each  of  these  factors  acting  in  a 
contradictory  manner. 

No  wonder  some  projects  have  an  overrun 
built  in  at  the  start.  The  requirements  are  too 
complex  and  sophisticated  for  the  govern- 
ment funds  budgeted  and  requested.  No  mat- 
ter what  sophisticated  management  tech- 


niques are  later  employed,  costs  and  schedules 
are  doomed  from  the  very  start. 

The  problem  of  insufficient  initial  funding 
is  compounded  by  the  auction  process  that 
often  goes  on  before  contract  award  called 
“best  and  final  offer.”  In  this  procedure,  con- 
tractor finalists  in  the  evaluation  process  are 
given  the  opportunity  to  make  a final  cost 
bid.  The  contractors  are  placed  in  the  terrible 
dilemma  of  whether  or  not  to  destroy  the 
credibility  of  their  cost  estimates,  carefully 
conceived  during  the  bid-response  phase.  Fur- 
thermore, the  morale  of  the  entire  project 
team,  both  government  and  contractor,  is  af- 
fected if  the  contract  is  signed  at  a bid  price 
well  below  what  they  themselves  have  devel- 
oped as  a reasonable  range  of  cost  parameters. 
The  “best  and  final  offer”  procedures  places 
the  procurement  process  in  the  realm  of  a 
used  car  auction  and  runs  directly  contrary  to 
the  axiom  “you  get  what  you  pay  for.”  From 
the  start  this  procedure  acts  as  a force  to 
build  in  eventual  overruns. 

As  the  project  proceeds  various  routine  ap- 
provals can  often  be  delayed,  causing  costs  to 
increase  because  a large  work  force  is  on  the 
payroll.  Approval  delays  can  result  from  too 
detailed  approval  being  required  at  too  high  a 
level  within  the  Executive  Department  as  well 
as  Congressional  delays  where  budget  appro- 
priations do  not  come  through  on  the  sched- 
ule anticipated  in  the  original  project  plans. 

Throughout  the  life  of  a project,  the  acqui- 
sition process  is  subject  to  many  changes  that 
cause  delays,  most  of  which  in  turn  result  in 
increased  costs.  Budget  cuts,  reprogramming 
of  funds,  increasing  requirements  and  changes 
in  scope  or  technical  capabilities,  tend  to 
stretch  out  schedules  and  result  in  increased 
overall  costs. 

Continuity  of  management  within  a weapon 
system  project  is  difficult  to  maintain.  Not 
only  do  the  project  management  staffs  ro- 
tate, but  the  higher  echelons  within  the  De- 
partment of  Defense  often  have  an  even 
shorter  on-the-job  life. 

Because  of  military  rotation  requirements 
for  career  advancement,  it  is  not  in  the  best 
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interests  of  a military  project  manager  to  stay 
on  the  job  too  long.  This  is  compounded  by 
the  fact  that  the  project  manager  position 
does  not  itself  represent  an  optimum  career 
step  in  some  of  the  military  services.  This  is 
indeed  paradoxical,  for  the  weapons  the  mili- 
tary fight  with  are  as  important  as  how  the 
mihtary  use  the  weapons  and  with  how  many 
people.  Yet  many  capable  mihtary  officers 
try  to  avoid  project  management  positions  be- 
cause they  feel  these  positions  do  not  repre- 
sent a step  on  the  path  to  career  advancement 
to  the  top.  And  in  many  cases,  they  are  cor- 
rect. 

HISTORICAL  REMEDIES 

Historically,  the  Department  of  Defense 
has  attempted  to  focus  on  different  aspects  of 
program  management  at  different  times.  This 
has  resulted  in  emphasis  from  time-to-time  on 
concepts  such  as  PERT,  Value  Engineering, 
Zero  Defects,  Management  by  Objectives,  De- 
sign-to-Cost,  and  many  other  catch  words 
and  slogans.  Too  often  management  by  slo- 
gan leads  to  overemphasis  on  one  or  two  as- 
pects of  project  management  with  resultant 
defocusing  of  attention  on  other  equaUy  im- 
portant areas  that  can  cause  the  project  to 
fail. 

In  the  course  of  a project,  conscientious 
and  intensive  review  processes  are  developed 
so  that  all  echelons  are  aware  of  and  can  take 
action  on  the  project  and  its  management.  In 
many  cases  over-management  or,  as  it  is  often 
called,  micro-management  results.  One  sees 
micro-management  of  the  contractor  by  the 
service  and  its  project  office,  of  the  service  by 
the  Department  of  Defense,  and  of  the  pro- 
ject itself  by  Congress  and  the  Government 
Accounting  Office.  The  layers  of  intensive  re- 
view lead  to  excessive  reporting  at  all  eche- 
lons. This  results  in  an  increase  in  man  power, 
time,  and  costs. 

Since  the  contractor  has  the  implementa- 
tion responsibility,  most  of  the  burden  of  this 
excessive  reporting  falls  on  his  shoulders.  This 
is  in  addition  to  complex  control  procedures 
already  imposed  in  the  contract  so  that  the 
contractor  can  be  monitored  every  step  of  the 
way  down  to  the  lowest  level  of  work  activ- 


ity. Reports  cost  money  and  too  often  much 
information  generated  is  not  necessary  or 
used. 

RECOMMENDED  SOLUTIONS 

Many  of  these  deficiencies  have  been  recog- 
nized in  recent  years  by  key  defense  planners 
and  managers.  Many  initiatives  have  been 
taken  and  need  to  be  continued  and  rein- 
forced. Additional  steps  will  have  to  be  taken 
as  project  management  becomes  more  com- 
plex and  as  new  Congressional  and  budget 
review  procedures  take  effect.  Some  recom- 
mended actions  are; 

• Require  better  initial  planning  prior  to 
starting  the  weapons  system  procure- 
ment process  through  government  chan- 
nels which  should  eventually  be  reflected 
in  the  Request  For  Proposal. 

Improved  mission  definition  and  anal- 
ysis, coupled  with  clear  cut  technical  re- 
quirements, scheduling,  and  budgeting 
should  be  accomplished  during  the  con- 
ceptual stages  of  a weapon  system  pro- 
curement project. 

• Eliminate  contract  bidding  auctions. 

If  the  government  project  office  deter- 
mines schedules  and  cost  ranges  with 
confidence  before  initiating  a Request 
For  Proposal  they  are  in  a better  posi- 
tion to  evaluate  proposals  and  insure  the 
credibility  of  the  initial  contractor  esti- 
mate in  response  to  the  RFP.  With 
knowledge  of  the  reasonable  range  of 
schedules  and  costs,  the  contract  need 
not  be  auctioned  off  and  ridiculous  low- 
ball bids  can  be  spotted  immediately. 

• Good  project  managers  are  a key  to  suc- 
cessful weapon  system  procurement. 

In  this  most  important  phase  of  over- 
all defense  management  the  project  man- 
agement position  should  be  elevated  to 
one  of  prime  importance  and  distinc- 
tion for  career  mihtary  officers.  Military 
program  managers  should  be  able  to  stay 
on  the  job  long  enough  to  see  a major 
phase  of  a project  completed.  These  of- 
ficers should  be  able  to  spend  a major 
part  of  their  career  in  project  manage- 
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ment  and  still  have  an  opportunity  to 
succeed  to  the  top  positions  in  the  re- 
spective service.  The  Defense  Depart- 
ment has  taken  steps  to  provide  educa- 
tion for  project  managers  by  founding 
the  Defense  Systems  Management  Col- 
lege. This  institution  was  started  by  Da- 
vid Packard,  then  Deputy  Secretary  of 
Defense  and  has  been  encouraged  since 
by  his  successors.  The  results  of  their  ef- 
forts are  already  paying  off,  and  the  Col- 
lege should  be  given  increased  emphasis 
as  it  refines  the  curriculum  and  increases 
research  capabilities. 


Faster  and  more  orderly  budget  approval 
processes  are  needed  in  both  the  Execu- 
tive and  Legislative  branches  of  govern- 
ment. 

The  Congressional  Budget  and  Im- 
poundment Control  Act  of  1974  has 
many  implications  for  weapon  system 
procurement.  More  timely  and  long- 
range  estimates  are  required  by  the  Act 
that  also  defines  deadline  dates  for  Con- 
gressional fiscal  approvals.  As  a result, 
longer  looks  into  the  future  will  be  re- 
quired from  the  weapons  system  project 
coupled  with  the  promise  of  timely  Con- 
gressional budget  and  appropriations  ap- 
provals. Both  factors  will  tend  to  in- 
crease project  planning  stability  and  may 
even  lead  to  multiyear  funding  or,  at 
least,  earlier  agreements  on  projects  so 
that  smooth  development  and  produc- 
tion planning  can  be  accomplished.  This 
new  Congressional  Act  imposes  tighter, 
tougher  schedules  in  the  government  ap- 
proval process,  forcing  earlier  decisions 
as  weU  as  longer  projections. 


CONCLUSIONS 

There  is  promise  that  weapons  system  pro- 
curement can  be  managed  more  effectively 
and  efficiently.  Some  initiatives  and  new  con- 
cepts are  underway.  Much  depends  on  how 
the  new  Administration  continues  and  rein- 
forces these  initiatives  and  seizes  the  manage- 
ment reins  to  develop  even  better  procedures 
and  personnel. 

There  should  be  increased  emphasis  on  im- 
proved, earlier  initial  planning  using  the  best 
resources  available  within  government  and  ob- 
tainable on  a contractual  support  basis. 

The  requirement  for  advanced  authoriza- 
tions and  early  information  to  Congress  under 
the  Budget  and  Impoundment  Act  will  place 
increased  emphasis  on  high  quality  estimates 
over  a longer  time  frame.  This  emphasis  is 
advantageous  but  will  increase  the  already 
heavy  work  load  of  the  weapons  system  pro- 
ject office. 

The  key  to  future  weapon  system  pro- 
curement success  lies  within  the  various  ech- 
elons of  the  Department  of  Defense  and  the 
military  services.  Not  only  should  DOD  and 
the  military  services  be  manned  with  suffi- 
cient, well-qualified  people,  but  they  should 
be  given  full  support,  in-house  and  contract- 
ually to  accomplish  their  important  manage- 
ment functions  and  processes. 

The  focal  point  of  future  systems  acquisi- 
tion success  or  failure  will  lie  increasingly  in 
the  military  project  offices.  These  offices 
should  be  given  the  capability,  -responsibility 
and  authority  to  carry  out  the  function  of  ac- 
quiring major  weapons  systems  on  time  and 
on  budget.  D 
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A New  Dimension 

In  the  Acquisition  Process 

by 

Jacques  S.  Gansler 

Deputy  Assistant  Secretary  of  Defense  (Materiel  Acquisition) 


In  this  article  the  author  challenges  the  reader  to  think  of  the  Defense  system  acquisi- 
tion process  from  a perspective  different  from  that  which  is  customary.  The  questions  asked 
are;  “Has  the  solving  of  individual  problems  resulted  in  a piecemeal  approach?”  “Has  sub- 
optimization  in  selected  areas  been  a major  contributor  to  the  long  time  lapse  from  program 
initiation  to  the  fielding  of  equipment(s)  the  program  was  designed  to  produce?” 


In  the  past  10  years  there  have  been 
major  efforts  made  to  try  to  get  a handle 
on  weapons  systems  acquisition  costs  and  risks 
and  there  has  been  considerable  success  in 
both  areas.  However,  I am  concerned  that  the 
approach  to  solving  each  of  the  individual 
problems  that  arose  was  perhaps  too  “piece- 
meal.” Each  problem  was  addressed  with  a 
“tailored  solution”  which,  in  fact,  did  tend  to 
reduce  cost  in  the  specific  area  and  did  fre- 
quently reduce  the  risk  at  the  same  time. 
However,  now  it  is  time  to  step  back  and  as- 
sess the  overall  acquisition  process  again.  I 
submit  that  we  may  have  “suboptimized”  in 
many  compartmentalized  areas,  and  that  we 
have  not  recognized  the  inherent  conflicts  and 
contradictions  between  and  among  these  se- 
lected areas.  The  net  effect,  I fear,  has  been 
the  current  very  long  and  very  expensive  over- 
all process-the  long  time  from  initiation  of  a 
program  to  its  deployment.  It  is  to  this  latter 
problem  that  I call  your  attention. 

In  the  past  several  years  we  have  begun  to 
recognize  some  of  the  inherent  conflicts  in 
the  acquisition  process,  and  to  take  some  cor- 
rective actions.  For  example,  in  the  past,  per- 


* Adapted  from  an  address  to  the  Conference  on  Tac- 
tical Missiles,  National  Bureau  of  Standards,  27  April  1977. 


formance  and  development  costs  were  empha- 
sized as  the  driving  factors  in  weapons  sys- 
tems acquisition.  When  we  were  done,  it  was 
found  that  the  systems  developed  were  too 
expensive  to  produce  in  the  necessary  quanti- 
ties, or  too  sophisticated  and  complex  to  be 
supported  in  the  field.  The  need  for  designing 
in  producibility  at  low  cost  in  design-to-cost 
efforts,  while  at  the  same  time  embracing  re- 
duced operating  and  support  costs  goals  is 
recognized.  However,  there  is  the  inherent 
conflict  between  minimizing  operating  and 
support  costs  and  maximizing  readiness.  Lit- 
tle is  done  in  the  development  cycle  today  to 
explicitly  address  readiness.  What  is  needed, 
and  what  we  have  begun  to  achieve,  is  the  in- 
tegration of  a production  and  support  per- 
spective into  the  weapon  system  development 
phase  of  our  programs.  Clearly,  this  is  not  a 
case  of  giving  something  to  everyone.  In  fact, 
tradeoffs  must  be  made  to  achieve  Defense 
objectives  within  the  available  resources. 
Thus,  the  last  few  percent  of  performance 
may  have  to  be  sacrificed  for  reduced  produc- 
tion and  support  cost  or  improved  readiness. 

If  one  were  to  step  back  and  look  at  the  to- 
tal acquisition  cycle  and  the  changes  that  have 
been  brought  about  over  the  past  10  years, 
one  could  say  that  concurrency  has  been 
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largely  eliminated  and  that  significant  steps 
have  been  taken  towards  reducing  risks.  How- 
ever, we  have  added:  “incremental  decision- 
making,” an  increase  in  management  reviews, 
considerable  increase  in  test  and  evaluation, 
competitive  prototyping,  low-rate  initial  pro- 
duction etc.  Each  of  these  additions  had  the 
desired  effect,  but  has  also  greatly  increased 
the  acquisition  cycle  time  and  cost.  We  have 
not  removed  anything!  The  effect  has  been 
that  where  we  were  able  to  field  the  NIKE 
AJAX  in  6 years  and  the  HAWK  in  5 years, 
from  requirement  to  deployment,  it  is  likely 
to  take  19  years  to  field  the  PATRIOT  and  18 
years  to  field  the  AEGIS. 


Let  me  briefly  cover  some  of  these  steps 
so  that  you  may  see  some  of  the  actions  al- 
ready taken  in  this  direction,  and  so  a better 
reference  for  the  subsequent  discussion  on  ad- 
ditional steps  may  be  established.  These  addi- 
tional steps  are  needed  to  specifically  address 
the  question  of  how  to  shorten  the  acquisi- 
tion cycle. 


INTEGRATION  OF 
DEVELOPMENT  AND 
PRODUCTION  PLANNING 


Finding  ways  to  compress  the  cycle  with- 
out increased  concurrency  or  risks  is  one  of 
our  most  difficult  challenges  for  the  near  fu- 
ture. 

There  are  two  major  conceptual  approaches 
that  should  be  taken  to  address  this  problem. 
First,  I submit  we  need  to  do  much  more 
early  planning  of  the  entire  acquisition  cycle. 
This  includes  the  normal  development  cycle 
process,  and  the  planning  of  alternatives,  de- 
cision options,  deviations  from  normal  prac- 
tices, acquisition  strategies,  industrial  base  im- 
pacts, production,  maintenance  etc. 


Early  planning  includes  considerably  more 
“what  if’  planning;  such  that,  when  an  event 
occurs  we  don’t  wait  6 to  9 months  for  the 
decisionmakers  to  evolve  a plan  for  the  next 
step.  Early  planning  includes  far  more  “key 
decision”  points  during  the  cycle.  If  signifi- 
cant achievements  can  be  realized  early,  or  if 
problems  develop  during  early  testing,  new 
directions  can  be  taken  to  minimize  costs  and 
time. 

Secondly,  I assert  that  far  too  much  has 
been  made  of  the  differences  between  De- 
fense acquisitions  and  commercial  practices. 
A move  in  the  direction  of  greater  similarity 
would  be  extremely  beneficial  to  the  Depart- 
ment of  Defense.  The  steps  that  we  have 
taken  over  the  past  2 or  3 years  are  steps  in 
the  direction  of  bringing  military  and  com- 
mercial practices  closer  together. 


About  70  percent  of  our  total  weapon  sys- 
tem acquisition  and  support  costs  are  essenti- 
ally determined  during  the  conceptual  stages 
of  equipment  development.  Because  of  this  it 
is  imperative  that  the  necessary  kind  of  atten- 
tion be  focused  at  the  front  end  of  the  proc- 
ess, to  reduce  “downstream  costs.”  Some  on- 
going initiatives  are  discussed  here. 

First,  more  attention  is  being  given  to  the 
initial  stages  of  the  acquisition  process.  As  di- 
rected by  0MB  Circular  A- 109,  entitled 
“Major  System  Acquisitions,”  the  mission 
need  is  evaluated  more  critically  and  a wider 
range  of  available  technologies  to  meet  that 
need,  quickly  and  efficiently-are  considered 
both  in  terms  of  performance  and  life-cycle 
costs. 

A major  initiative  is  to  improve  and  strength- 
en in-house  production  planning.  It  is  ironic 
in  a sense  that  although  our  production  ac- 
count varies  between  65  percent  and  80  per- 
cent of  acquisition  cost,  in  many  cases  the 
production  community  has  no  involvement 
until  most  of  the  parameters  that  influence 
these  costs  have  been  determined.  The  pro- 
duction community  must  become  involved 
earlier  in  the  systems  acquisition  process  to 
ease  the  transition  from  development  into 
production  and  to  help  reduce  total  cost.  To 
accomplish  this,  we  stress  production  plan- 
ning assessments  early  in  the  development  cy- 
cle, and  urge  early  identification  of  manufac- 
turing technology  voids  to  aid  the  transition 
process. 

In  addition  we  are  looking  more  to  proto- 
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type  competition  for  technological  innova- 
tion-to  obtain  performance  improvements, 
cost  reduction  and  risk  avoidance.  I think  we 
can  use  this  tool  more  efficiently  than  we 
have.  Significant,  qualitative  improvements 
from  innovation  are  more  likely  to  be  found 
in  the  prototype  development  phase  than  dur- 
ing competitive  production,  where  quantita- 
tive changes  only  are  likely. 

Design-to-cost  is  a commercial-product 
management  tool  used  to  help  integrate  the 
development  and  production  phases.  The  con- 
cepts of  design-to-unit-production  cost  and 
design-to-minimum-life-cycle  cost  are  becom- 
ing institutionalized— goals  are  being  estab- 
lished for  both  parameters  early  in  the  devel- 
opment cycle.  Design-to-cost  techniques  have 
contributed  to  reducing  the  rate  of  noninfla- 
tion cost  growth  of  systems  (as  identified  in 
our  SAR  reports  to  Congress)  from  over  6 
percent  per  year  in  1973  (when  we  were  get- 
ting started),  to  around  3 percent  today.  The 
challenge  is  to  keep  moving  in  this  positive  di- 
rection as  more  flexibility  and  visibility  is 
built  into  the  management  process. 

The  new  Source  Selection  Directive  is  a 
major  step  in  the  direction  of  lower  cost 
systems.  This  Directive  states  that  develop- 
ment awards  will  be  made  based  upon  the  in- 
herent production  and  support  cost  of  the 
proposed  system— not  primarily  on  the  pro- 
posed development  program  cost. 

Even  though  the  Department  of  Defense 
has  the  responsibility  of  providing  the  “re- 
quirements” framework  for  new  systems,  in- 
dustry’s hands  cannot  be  tied  if  lower  cost 
systems  are  truly  desired.  We  want— even  dur- 
ing the  bidding  process— Request  for  Proposal 
(RFP)  and  contract  changes  recommended  to 
us  that  will  be  cost  effective.  The  new  Source 
Selection  Direction  also  takes  a step  in  this  di- 
rection. 

A chronic  complaint  from  industry  has 
been  that  over-application  of  military  specifi- 
cations and  standards  drives  costs  up.  There- 
fore, a major  effort  is  presently  underway  to 
review  all  40,000-1-  specifications  and  stand- 
ards. The  goal  is  to  eliminate  unnecessary 
specifications  and  standards  and  to  update 


those  that  need  it;  but,  most  important,  to 
allow  and  encourage  “tailoring”  to  individual 
program  requirements.  Rather  than  starting 
by  issuing  directives  that  dictate  the  “scrub- 
bing” of  Requests  for  Proposals  and  the  “tail- 
oring” of  specifications  and  standards,  a con- 
siderable amount  of  missionary  work  was  per- 
formed in  this  area.  As  a result,  major  pro- 
grams such  as  HELLFIRE,  the  Navy  Elec- 
tronics Warfare  Suite,  and  the  F-18  have  al- 
ready tailored  various  requirements. 

The  objective  is  to  get  the  Services  to  cause 
the  necessary  “cultural  change”  to  take  place, 
and  I believe  a good  start  has  been  made.  The 
Services  have  issued  some  good  “implemen- 
tors.” Based  upon  these  efforts  and  other 
“lessons  learned,”  an  overall  “scrubbing  and 
tailoring”  policy  has  been  stated  in  the  new 
DOD  Directive  4120.21  entitled,  “Specifica- 
tions and  Standards  Application.” 

Just  to  mention  some  of  the  other  initia- 
tives for  which  new  policy  guidance  is  being 
issued:  Commercial  specifications  are  being 

adopted,  greater  use  of  commercial  equip- 
ment and  product  warranties  is  being  made, 
software  is  being  standardized,  and  revitaliza- 
tion of  the  Value  Engineering  program  is  in 
progress. 

Actions  designed  to  improve  the  efficiency 
of  our  production  process  complement  these 
cost  reduction  efforts.  Manufacturing  technol- 
ogy should  be  emphasized  early,  in  concert 
with  overall  front-end  production  planning,  to 
help  bridge  the  gap  between  development  and 
production.  The  manufacturing  technology 
program  has  been  redirected,  to  first  identify 
production  cost  drivers  that  need  attention; 
then  to  provide  “seed  money”  to  assist  indus- 
try in  developing  innovative,  and  less  costly. 
Defense  production  methods.  Annual  DOD 
funding  in  this  area  is  being  doubled— to  over 
$200  million.  A recent  success  story  in  this 
area  is  the  GAU  8 ammunition-through  a $3 
million  investment  in  a new  manufacturing 
process,  $300  million  in  production  costs 
were  avoided. 

Another  area  to  be  attacked  is  the  obsoles- 
cence of  plants  and  equipment  in  the  Defense 
industry.  As  a result  of  the  “Profit  76”  study 
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the  DOD  profit  policy  has  been  revised  to  re- 
ward needed  investment  by  making,  for  the 
first  time,  the  imputed  interest  cost  of  facili- 
ties an  allowable  overhead  cost.  Also,  the 
weighted  guidelines  have  been  changed  to  pro- 
vide increased  profit  based  on  company  capi- 
tal investment.  Concurrently,  other  invest- 
ment incentives  are  under  consideration  to  en- 
courage industrial  modernization;  these  in- 
clude greater  use  of  multiyear  contracting 
and  special  termination  provisions  to  reduce 
risk. 

To  move  in  the  direction  of  commercial 
practices,  day-to-day  involvement  in  the  con- 
tractor’s activities  must  be  reduced.  As  a step 
in  this  direction,  almost  2,500  people,  who 
performed  quality  and  contract  administra- 
tion functions  are  being  removed  from  con- 
tractor plants. 


On  the  industrial  base  side  the  idea  of  com- 
partmentalizing, or  suboptimizing,  applies 
equally  well.  The  past  approach  to  conduct- 
ing business  with  the  Defense  Industry  has 
been  to  focus  attention  on  specific  cost  ele- 
ments at  the  prime  contractor  level.  As  a re- 
sult, a “dual  economy,”  with  differing  market 
characteristics,  has  been  developed  at  the 
prime  contractor  and  the  subcontractor/ 
parts-supplier  levels. 


labor  stability  (e.g.,  the  “constant  work 
force”  concept). 


EARLY  OPERATING  AND 
SUPPORT  PLANNING 

The  conflicts  and  challenges  faced  in  trying 
to  minimize  the  cost  to  operate  the  support 
equipments  while  improving  force  readiness 
are  myriad. 

I am  sure  that  you  have  been  made  aware 
of  the  trends  regarding  the  DOD  procurement 
account.  In  the  decade  since  the  Vietnam 
peak.  Defense  buying  power  has  experience  a 
drastic  erosion,  going  from  a high  in  1968  of 
$47  billion  in  outlays  (in  constant  77  dollars) 
to  approximately  $28  billion  this  year  (1977), 
having  bottomed  out  in  1975  at  about  $17 
billion.  Over  that  same  period  of  time  a signi- 
ficant growth  in  the  share  of  the  budget  going 
for  operations  and  support  was  experienced. 
However,  even  with  the  cost  growth,  readiness 
problems  are  experienced.  Notable  too  is  the 
fact  that  the  interest  of  the  Congress  in  the 
readiness  of  our  military  forces  has  picked  up 
considerably  over  the  past  few  months. 


For  example,  a joint  DOD/OMB  study  of 
the  US  aircraft  industry  was  recently  com- 
pleted. The  chief  findings  were  that  this  in- 
dustry, at  the  prime  contractor  level,  is  op- 
erating at  about  55  percent  of  one-shift 
capacity,  and  the  cost  of  the  idle  capacity  is 
conservatively  costing  the  Defense  Depart- 
ment on  the  order  of  $400  million  per  year. 
Similarly,  it  was  found  that  “bottlenecks” 
for  production  “surge”  rest  primarily  at  the 
parts-supplier  level— and  this  base  is  shrinking 
rapidly.  Corrective  actions  have  been  initiated 
at  both  levels.  The  main  point  is  that  a 
broader  perspective  is  being  taken. 

Similarly,  where  previous  attempts  were 
made  to  optimize  the  development  timing  and 
production  rate  for  each  program,  we  now 
look  at  the  overall  industry  sector  and  firm 
impacts,  and  consider  program  timing  and 


The  Department  of  Defense  is  trying  to  do 
something  about  readiness,  and  simultane- 
ously reduce  the  fraction  of  the  DOD  budget 
allocated  to  operating  and  support  costs— a 
tough  challenge! 

Background;  The  Department  of  Defense 
spends  approximately  $14  billion  annually  to 
buy  spares  and  consumables  to  support  US 
operational  forces.  Even  though  Secretary 
Brown’s  amended  Fiscal  Year  1978  budget 
called  for  a net  $2.8  billion  reduction  in  other 
areas,  more  than  $600  million  was  added  back 
for  readiness  improvements.  This  approach 
alone  cannot  solve  the  problems.  Long-term 
strategy  must  get  the  US  back  in  the  position 
where  procurement  constitutes  a larger  per- 
centage of  the  budget  than  operations. 

The  DOD  is  working  to  get  the  parameters 
which  impact  readiness  and  associated  operat- 
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ing  and  support  costs  defined  early  in  the  de- 
velopment cycle.  From  this,  a cost-effective 
set  of  equipment  design  and  logistics  support 
alternatives  which  are  consistent  with  our 
readiness  goals  can  be  developed.  Also  the 
DOD  is  looking  at  contracting  approaches- 
e.g.,  warranties— to  help  make  reliability  and 
maintainabihty  improvement  happen.  Clearly, 
followup  is  required  to  make  sure  that  the 
goals  are  met.  A radically  new  approach-a 
feedback  or  tracking  mechanism  that  follows 
the  equipment  into  deployment— may  be 
needed. 

SHORTENING  THE 
ACQUISITION  CYCLE 

To  return  to  overall  acquisition  cycle: 
While  technological  advances  are  occurring  at 
a more  rapid  rate— as  evidenced,  for  example, 
by  the  new  families  of  tactical  missile  guid- 
ance schemes,  the  acquisition  cycle  is  un- 
necessarily stretching.  Programs  such  as 
LANCE  and  the  TELEVISION  MAVERICK 
were  speedily  concluded  in  about  4 years.  To- 
day programs  such  as  COPPERHEAD  and 
HARPOON  may  exceed  8 years.  The  DOD 
needs  to  look  closely  to  see  if  this  trend  can 
be  reversed— without  ihcreased  risk. 

Historically,  the  length  of  the  acquisition 
cycle  has  been  perturbated  by  two  things- 
first,  disagreement  on  what  is  wanted,  and 
second,  the  tendency  to  bite  off  a larger  tech- 
nological chunk  than  we  are  capable  of  digest- 
ing. Circular  A- 109  forces  us  to  resolve  the 
first  item,  and  Secretary  Brown’s  recent  pol- 
icy statements  emphasizing  simplicity  and  re- 
liability as  weapon  goals  requires  us  to  face 
squarely  the  second.  Given  these  two  steps, 
the  decision  process  must  be  revised  to  take 
advantage  of  the  potential  for  more  rapid  de- 
velopments. 

The  risk,  cost,  performance  and  schedule 
tradeoffs  required  by  the  acquisition  manage- 
ment process  are  difficult  to  make,  at  best. 
However,  there  are  some  definite  steps  that 
can  be  taken  to  achieve  a better  tradeoff  bal- 
ance, without  undoing  the  positive  things  ac- 
complished to  date.  First,  a “hands-ofF’  ap- 
proach can  be  adopted  during  development 


for  programs  that  do  not  have  a high  degree 
of  technical  risk.  To  make  this  work,  two 
things  have  to  happen— first,  a requirements 
description  and  a test  plan  stating  what  is 
wanted  must  be  provided.  (Exclude  the  “how- 
to-design” specifications.)  Second,  a healthy 
competitive  environment  is  needed  to  get  in- 
novative ideas  and  offset  some  of  the  DOD 
risk  in  this  approach.  The  acquisition  cycle 
benefits  from  reduced  day-to-day  manage- 
ment and  suppliers  have  the  flexibihty  to  plan 
programs  in  the  most  time-efficient  manner. 
This  is  the  approach  now  being  tried  on  the 
“Air  Defense  Gun.” 

Next,  far  too  many  programs  that  incorpo- 
rate very  high  risk  subsystems  are  presented 
for  management  approval.  This  procedure 
provides  the  framework  for  a vicious  cycle  of 
test  and  redesign  between  the  subsystems  and 
the  carrier  vehicles.  Basically,  more  independ- 
ent feasibility  demonstrations  of  new  compo- 
nents and  new  technology  that  will  develop 
more  options  for  weapon  systems  are  needed. 
Less  full  scale  weapon  system  development 
should  be  done.  When  technology  is  fully 
demonstrated,  and  then  applied  to  a new 
system  or  a product  improvement,  costs  are 
reduced  by  almost  one-half. 

For  programs  involving  major  technological 
advancements  or  uncertainty  in  operational 
acceptability,  it  has  become  normal  practice 
in  the  last  few  years  to  develop  and  test  hard- 
ware prototypes  prior  to  entering  full  scale 
engineering  development.  The  A- 10,  F-16, 
XM-1  tank,  HARM  and  Imaging  Infrared  Mav- 
erick are  examples.  Perhaps  ways  to  make 
more  efficient  use  of  prototypes  should  be 
considered.  Obviously,  if  there  are  no  technol- 
ogy advancements  to  be  addressed,  proto- 
types are  not  required. 

If  prototypes  are  used,  they  should  be  used 
so  as  to  reduce  the  full  scale  development 
phase.  The  incremental  addition  of  system  re- 
quirements and  demonstration  objectives 
should  be  considered  as  the  prototypes  be- 
come stronger  candidates  to  fill  a particular 
mission  need.  With  these  incremental  addi- 
tions the  overall  cycle  can  be  shortened.  A 
start  from  scratch  in  the  full  scale  develop- 


10 


Defense  Systems  Management  Review 


merit  phase  would  not  be  required  nor  would 
it  be  necessary  to  repeat  successful  tests. 

The  DOD  prototype  programs  are  struc- 
tured as  low-cost  programs— minimum  draw- 
ings, modified  qualifications  etc.  As  the  pro- 
totype program  evolves  into  a major  contend- 
er, development  activities  can  be  modified, 
and  production  planning  and  producibility  a- 
nalysis  can  be  instituted.  At  this  point  it  may 
be  possible  to  skip  full  scale  engineering  devel- 
opment and  enter  a low-level  production 
phase. 

Another  fertile  area  for  possible  time  sav- 
ing is  the  time  allocated  for  test  and  evalu- 
ation. The  earlier  testing  begins  the  better; 
problems  are  found  at  a point  in  time  when 
they  can  be  solved  with  less  cost  and  schedule 
impacts.  Many  of  the  programs  today  have 
separate  and  distinct  contractor,  developer 
and  user  test  phases.  Blending,  or  at  least  the 
sharing  of  test  data  can  have  obvious  benefi- 
cial effects.  This  should  in  no  way  reduce  the 
independence  of  test  and  evaluation-but  it 
could  result  in  significant  time  savings. 

Another  area  to  achieve  time  compression 
is  part  of  the  Manufacturing  Technology  Pro- 
gram; i.e..  Computer  Aided  Design  (CAD). 
The  automotive  industry  states  that  CAD  has 
reduced  the  development  time  for  its  cars 
from  over  2 years  down  to  5 months.  The  be- 
lief is  that  design  changes  using  CAD  can  be 
made  relatively  risk  free.  The  transition  from 
CAD  to  Computer  Aided  Manufacturing 
(CAM),  using  the  same  software,  also  contrib- 
utes to  a greatly  reduced  overall  cycle.  The 
CAD/CAM  arrangement  permits  going  di- 
rectly from  development  into  production. 
The  challenge  is  to  see  if  some  of  the  CAD/ 
CAM  potential  benefits  can  be  shared  by  the 
Defense  Industry. 

An  obvious  shortcut  in  the  acquisition 
process,  and  at  low  risk,  is  through  improve- 
ments to  existing  equipment.  Improved 
HAWK  was  a good  step  in  this  direction,  as 
well  as  the  new  seekers  for  our  air-to-air  and 
air-to-ground  missile  series.  In  the  future,  the 
DOD  may  be  forced  to  this  approach  for 
fiscal  reasons  as  much  as  to  satisfy  the  desire 
to  field  a capability  at  an  earlier  date. 


North  Atlantic  Treaty  Organization  stand- 
ardization is  being  given  top  priority  atten- 
tion. Here,  the  objectives  are  to  improve  force 
interoperability,  make  better  use  of  total  Al- 
lied resources,  and  lower  costs  of  develop- 
ment, acquisition  and  logistics  support.  Allies 
of  the  US  have  made  it  clear  that  if  standard- 
ization is  to  really  work,  it  must  not  simply 
mean  everyone  purchasing  the  US  systems. 
The  Allies  want  the  US  to  be  open  to  purchas- 
ing their  weapons.  The  DOD  policy  is  consis- 
tent with  the  desires  of  our  NATO  Allies.  The 
DOD  will  buy  weapons  developed  by  our  Al- 
lies when  these  weapons  meet  US  needs  and 
are  cost  effective.  The  US  will  foster  interop- 
erability and  standardization.  Standardiza- 
tion, through  the  use  of  “off  the  shelf ’for- 
eign systems,  offers  both  time  and  cost  sav- 
ing possibihties-if  it’s  done  right! 

Also,  the  US,  with  a gentle  nudge  from  leg- 
islation such  as  the  Culver-Nunn  Amendment, 
is  considering  codevelopment  programs  to  a- 
chieve  NATO  standardization.  A joint  devel- 
opment program,  between  DOD  and  NATO 
Allies,  would  have  reduced  the  acquisition  cy- 
cle for  joint  programs  such  as  ROLAND  and 
AWACS.  The  US  could  have  avoided  the  sec- 
ond development  iteration  evident  in  both 
cases. 

A thought  to  consider  on  shortening  the  ac- 
quisition cycle  is  the  concept  of  “parallel  de- 
cisionmaking.” Here,  the  idea  is  to  have  status 
monitoring  of  periodic  and  significant  events 
and  to  issue  incremental  decisions-often 
based  on  “what  ifs.”  The  cycle  length  and  the 
risk  can  be  reduced  by  periodic  releases, 
rather  than  waiting  for  major  milestones  that 
can  be  years  apart.  Incremental  decisions 
would  also  eliminate  the  6 to  9 months  it  of- 
ten takes  for  decisionmaking  at  these  “gates.” 
The  actions  could  be  planned  well  in  advance, 
so  the  incremental  decisions  and  releases 
would  be  consistent  with  options  contained  in 
the  long-range  plans.  The  DOD  is  struggling 
with  the  “hows”  of  making  this  idea  work. 
Your  help  and  views  on  this  matter  are  ear- 
nestly solicited. 

There  is  no  easy  “cook  book”  solution  to 
obtaining  a good  balance  between  risk,  perfor- 
mance, cost  and  schedule  in  acquisition  pro- 
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grams.  To  shorten  the  acquisition  cycle,  a va- 
riety of  things  can  be  considered;  these  in- 
clude : 

• Possible  organizational  changes  to  get 
our  thoughts  on  a more  aggregated  level. 

• A reconsideration  of  the  decision  mak- 
ing process  toward  more  preplanned, 
incremental  decision  points. 

• Changes  in  contractual  procedures  to 
provide  stimulus  for  suppliers  to  furnish 
the  best  product  in  the  shortest  time. 

• Through  front-end  planning  to  tie  the 
process  together. 

SUMMARY 


In  summary,  I suggest  that  the  acquisition 
process  has  been  approached  in  too  much  of 
a piecemeal  fashion.  Two  necessary  actions 
can  help  bring  the  total  acquisition  process 


back  to  proper  perspective.  First,  through 
front-end  planning,  a more  encompassing  view 
must  be  taken  toward  optimizing  the  com- 
plete cycle.  This  forces  a recognition  of  con- 
flicts between  subelements  and  promotes  ef- 
fective early  tradeoffs.  Secondly,  we  must  ap- 
ply policies  and  procedures  which  are  more 
consistent  with  commercial  practices-e.g., 
design-to-cost,  “hands-off’  competition,  war- 
ranties and  “tailoring”  of  specifications. 

The  problem  is  to  translate  these  broad 
policies  into  workable,  contractual  agree- 
ments between  the  DOD  and  its  industrial 
suppliers. 

I have  tried  to  provide  you  with  some 
thoughts  on  DOD  initiatives  to  improve  the 
acquisition  process.  I have  also  provided  you 
with  some  ideas  on  how  an  excessively  lengthy 
acquisition  cycle  might  be  reduced,  with- 
out increased  program  risk.  This  is  the  chal- 
lenge I leave  with  you.  Help  us  solidify  our 
thoughts  into  workable  concepts  and  then 
help  us  (DOD)  implement  them.  □ 


Mr.  Gansler  is  Deputy  As- 
sistant Secretary  of  Defense 
(Materiel  Acquisition).  Prior 
to  this  assignment  he  was  As- 
sistant Director  of  Defense  Re- 
search and  Engineering,  first 
responsible  for  Electronics  and 
later  for  R&D  Planning. 

Before  joining  the  Office  of  the  Secretary  of  Defense,  Mr. 
Gansler  was  Vice  President  and  Director  of  Business  Develop- 


ment, Avionics  Division,  International  Telephone  and  Tele- 
graph Corporation.  Earlier  he  held  management  positions 
with  the  Singer  Company  and  the  Raytheon  Corporation. 

Mr.  Gansler  has  a B.E.  from  Yale  University,  an  M.S.  from 
Northeastern  University  (Electrical  Engineering),  an  M.S.  in 
Political  Economy  from  the  New  School  for  Social  Research 
and  has  completed  his  course  work  for  a Ph.D.  in  Economics 
at  American  University.  He  has  been  an  instructor  in  servo- 
mechanisms, and  has  published  numerous  professional  pa- 
pers. 


12 


Defense  Systems  Management  Review 


Tailoring 

Program  Requirements 

by 

William  F.  Brown 
Program  Coordinator 


The  author  has  engaged  in  the  tailoring  of  requirements  for  proposals  and  programs  during 
all  phases  of  the  program  life  cycle. 

Most  of  the  information  presented  in  this  article  is  based  upon  a paper  that  was  sponsored 
by  the  author  for  the  Technical  Management  Committee,  Aerospace  Industries  Association. 
That  paper  included  suggestions  of  aU  Committee  Members.  Thus,  what  is  presented  here 
represents  an  industry  position  on  the  subject.  The  information  is  consistent  with  published 
military  procurement  pohcies. 


DECISION  TO  “TAILOR” 

Department  of  Defense  interest  in  the  ap- 
plication of  specifications  and  standards 
became  manifest  in  November  1974.  At  that 
time  the  Defense  Science  Board  was  asked  to 
estabUsh  a Task  Force  to  identify  the  factors 
contributing  to  unnecessary  contract  costs 
arising  from  mihtary  specifications  and  stand- 
ards. Appropriate  action,  to  be  implemented 
through  Department  of  Defense  (DOD)  di- 
rectives and  instructions,  was  to  be  recom- 
mended. The  Task  Force  consisted  of  both 
industry  and  military  department  executives. 
The  Chairman  was  Dr.  J.F.  Shea,  Senior  Vice 
President  of  the  Raytheon  Company.  An  in- 
terim report  of  the  Task  Force  prompted  the 
then  Deputy  Secretary  of  Defense,  W.P.  Cle- 
ments, to  issue  a memorandum  for  the  secre- 
taries of  the  military  departments  entitled 
“Specification/Standards  Application.”  The 
memorandum  was  dated  4 August  1975.  In 
tins  document  Mr.  Clements  indicated  that 
his  staff  had  been  instructed  to  initiate  appro- 
priate procedures,  regulations  and  policies  and 


to  implement  measures  to  correct  the  prob- 
lems identified  by  the  Defense  Science  Board 
Task  Force.  The  memorandum  specifically 
stated  that  specifications  and  standards 
should  not  be  contractually  invoked  without 
a specific,  coordinated  “scrub  and  tailor” 
process. 

The  tailoring  of  requirements  through  the 
life  cycle  of  a program,  was  recommended  by 
the  Task  Force  on  Specifications  and  Stand- 
ards of  the  Defense  Science  Board,  and  is  now 
emphasized  by  both  Government  and  by  Indus- 
try. The  military  procuring  activities  have 
begun  to  actively  promote  tailoring  and 
Armed  Services  Procurement  Regulation, 
ASPR  1-1201,  has  been  revised  to  reflect  this 
interest.  The  tailoring  of  requirements  re- 
quires consideration  from  the  moment  a need 
to  perform  an  essential  mihtary  mission  is 
evidenced  and  the  wherewithal  to  conduct  the 
mission  is  first  being  thought  about.  At  this 
time  the  process  of  sorting  out  specifications 
and  standards,  or  portions  thereof,  that  will 
eventually  be  applicable,  begins. 


Vol.  I,  No.  4 


13 


DEFINITION  OF  TAILORING 

The  tailoring  of  requirements  is  an  orderly 
process  of  sorting  general  military  specifica- 
tions into  a specific  set  of  requirements  that 
when  modified  will  be  applicable  to  a certain 
procurement.  General  military  specifications 
are  available  for  innumerable  products,  serv- 
ices and  documentation.  The  type  of  product 
selected  to  perform  the  mission  is  first  taken 
into  account.  Next  the  phase  of  the  acquisi- 
tion cycle  is  determined  and  the  end  use  of 
the  item.  The  tailoring  of  the  requests  takes 
place  in  three  steps: 

• A gross  selection  of  specifications/stand- 
ards to  be  applied. 

• Deletion  of  paragraphs  containing  re- 
quirements which  are  not  to  be  applied. 

• Adjusting  the  remainder  of  the  require- 
ments to  suit  the  procurement  and  docu- 
menting the  results. 

Requirements  (existing  applicable  specifica- 
tions and  standards)  are  arranged  and  col- 
lected in  major  categories  so  that  they  can  be 
applied  through  the  statement  of  work  or 
suitable  contract  addenda  to  a request  for 
proposal  and  to  the  eventual  contract  in  an 
organized  manner. 


GENERAL  PRACTICES 

There  have  always  been  varying  degrees 
of  tailoring  in  all  phases  of  system  acquisition. 
Some  requirements  have  been  carefully  tai- 
lored for  use  with  a request  for  proposal  but 
most  have  not.  One  example  of  an  office  that 
regularly  performs  careful  tailoring  is  the  Sys- 
tems Criteria  Branch  of  the  Naval  Air  Systems 
Command.  This  office  creates  a type  specifi- 
cation and  proposed  contract  addenda  for  ma- 
jor categories  of  subjects  to  be  applied  to  pro- 
grams. Possibly  other  offices  are  equally  well 
organized  in  this  regard,  but  the  typical  prac- 
tice has  been  to  issue  a Request  for  Proposal 
(RFP)  with  a wholesale  collection  of  all  the 
possible  requirements  applied  in  full.  Many  of 
the  documents  applied  include  references  to 
lower  tier  specifications  and  standards  whose 


application  is  not  defined.  The  contractor  is 
then  faced  with  the  problem  of  responding  to 
the  stated  requirements  even  though  he  is 
virtually  certain  that  a large  percentage  are 
either  not  necessary  or  are  not  applicable  to 
the  program  phase.  The  subsequent  contract 
award  is  the  beginning  of  a sorting  process  to 
eliminate  and  adjust  these  requirements. 

The  sorting  process  can  consume  years 
through  several  program  phases  by  continual 
negotiation  and  deviation.  Meanwhile,  the 
product  is  relatively  unaffected  as  it  wends 
its  way  to  completion,  largely  obhvious  to  the 
paper  shuffle  that  contributes  httle  to  its  ex- 
cellence but  which  detracts  seriously  from  the 
mainstream  of  effort.  The  process  is  fraught 
with  pitfalls,  not  the  least  of  which  is  the  lack 
of  proper  program  task  definition  and  the 
consequent  lack  of  a firm  basis  for  the  pro- 
gram cost  estimate.  One  major  reason  for  this 
deficiency  is  the  lack  of  a suitable  method  for 
organizing  the  requirements. 

ORGANIZATION  OF  SUBJECTS 

For  a typical  contract,  most  requirements 
can  be  said  to  fall  within  the  following  main 
categories: 

Product  design  requirements 
Management  systems 

Cost  and  schedule  control  system 

Configuration 

Assurance 

Safety 

Test  and  demonstration 
Integrated  logistics  support 
Packaging  and  delivery 
Documentation 
Terms  and  conditions 

The  relation  of  the  above  can  be  seen  in 
Figure  1 . The  relation  of  these  items  to  a re- 
quest for  proposal  and  to  a contract  is  nearly 
identical. 

PERFORMING  TAILORING 

There  are  known  means  for  tailoring  re- 
quirements. The  process  requires  painstaking 
work  and  organization.  The  level  at  which  tai- 
loring must  take  place  is  between  the  contract 
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Figure  1.  Progression  of  Tailoring  Process  (from  general  requirements  to  contract  for  a typical  procuring 


level  and  the  vast  stratum  of  existing  specifi- 
cations, standards,  handbooks  and  manuals 
written  around  every  conceivable  subject 
from  chewing  gum  to  complex  firecontrol  sys- 
tems and  complete  aircraft  and  missiles:  The 
pOD  index  has  75  on  a typical  page  and  the 
index  contains  about  550  pages.  The  process 
described  assumes  that  every  document  men- 
tioned, directly  or  indirectly,  in  the  RFP  is 
either  in  the  possession  of  the  contractor  or  is 
at  least  available  to  him  for  review. 


Application  of  Documents 

At  least  eighty  percent  of  indexed  military 
documents  are  inherently  not  applicable  to 
any  given  procurement  by  virtue  of  subject 
alone.  Some  areas  of  procurement  are  even 
more  fortunate.  Propulsion  devices  for  many 
years  have  had  the  applicability  of  specifica- 
tions and  standards  limited  to  only  an  essen- 
tial few  by  the  issuance  of  a Government  bul- 
letin which,  in  effect,  performs  tailoring  for  a 


Vol.  I,  No.  4 


15 


particular  product  on  a broad  basis,  and  has 
been  markedly  successful  in  its  application 
and  use. 

Limited  Application 

Some  contractors  have  been  successful  in 
negotiating  individual  contracts  in  which  the 
mandatory  requirements  were  limited  to  the 
first  tier  of  applicable  documents.  This  al- 
lowed the  remainder  of  the  documents  to  be 
used  as  guides  only  and  did  not  subject  these 
documents  to  the  specific  deviation  proce- 
dure. Rather  than  limit  this  approach  to  a few 
isolated  applications,  it  is  worthy  of  consider- 
ation for  universal  adoption. 

Organization 

Successful  tailoring  of  a series  of  require- 
ments documents  for  a procurement  is  an  or- 
ganized, thought-provoking  refinement  proc- 
ess. The  process  requires  technical  expertise 
and  the  best  individual  attention  at  appropri- 
ate levels  of  management.  To  insure  adequate 
implementation,  the  process  must  be  under 
the  direct  cognizance  of  the  military  and  civil- 
ian program  managers. 

Existing  Procedures 

The  success  of  the  tailoring  process  requires 
that  there  be  an  existing  vehicle  at  the  level  of 
application  to  a Request  for  Proposal  or  con- 
tract that  can  be  adapted  to  the  purpose.  This 
vehicle  can  be  contract  addenda  or  a state- 
ment of  work,  or  both.  Reference  to  applica- 
ble military  documents  will  benefit  from  the 
organization  inherent  in  the  use  of  the  adden- 
dum and  statement  of  work  approach.  Herein 
lies  the  key  to  the  success  of  the  entire  endea- 
vor. The  tailoring  is  documented  at  this  level 
and  provides  the  tie  between  the  general  and 
detailed  military  requirements  and  the  spe- 
cific baseline  for  a given  procurement.  Docu- 
ments and  procedures  exist  for  this  purpose. 
New  methods  are  neither  needed  nor  recom- 
mended. 


RECOMMENDED  PROCEDURES 

A detailed  description  of  how  one  existing 


procedure  operates  on  a regular  basis  is  given 
here.  The  procedure  is  not  new  having  been  in 
use  for  at  least  30  years.  The  procedure  is  not 
perfect  but  has  been  largely  successful.  The 
process  involves  use  of  six  to  eight  documents 
designed  to  collect  requirements  in  broad  but 
related  categories  such  as  those  mentioned  un- 
der Organization  of  Subjects.  The  design  re- 
quirements for  the  hardware  are  first  tailored 
in  a “type”  specification  that  follows  the  for- 
mat of  an  existing  general  specification.  When 
used  with  the  request  for  proposal,  this  speci- 
fication provides  a place  to  document  the  re- 
quired missions  and  performance.  Each  part 
of  the  functional  organization  is  privileged  to 
provide  input.  The  specification  is  then  assem- 
bled by  persons  experienced  in  that  work.  En- 
gineering data  and  development  test  require- 
ments are  developed  and  assembled  in  a simi- 
lar fashion  by  the  preparation  of  an  adden- 
dum to  a general  specification  made  for  that 
purpose.  Demonstration  requirements  are  tai- 
lored in  the  same  way  as  are  the  requirements 
for  management  systems  and  logistics  sup- 
port. Data  requirements  are  collected  from 
these  sources  and  listed  on  a Contract  Data 
Requirements  List.  Thus  the  majority  of  the 
significant  tailoring  occurs  prior  to  the  re- 
quest for  proposal. 

Ongoing  Process  of  Refinement 

The  documents  tailored  in  the  manner  de- 
scribed are  transmitted  with  an  RFP  to  pro- 
spective contractors.  The  documents  are  fur- 
ther refined  by  each  contractor  in  his  pro- 
posal that  further  tailors  the  requirements  to 
a specific  program  and  proposed  system.  Most 
contractors  avoid  drastic  tailoring  at  this 
point  for  fear  of  being  found  nonresponsive. 
The  Government  must  be  encouraged  to  seek 
immediate  ways  to  overcome  this  obstacle 
since  tailoring  at  this  juncture  has  the  greatest 
potential  for  cost  savings  and  cost  avoidance. 
The  requirements  are  refined  still  further  to 
complete  the  predevelopment  tailoring  pro- 
cess during  contract  negotiation  so  that,  at 
the  time  of  contract  approval,  the  documen- 
ted results  are  available  as  contract  attach- 
ments. This  progression,  and  the  relation  of 
documents  are  shown  in  Figure  1.  The  process 
continues  into  the  subcontra ctor/vendor/sup- 
plier  relations. 
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Responsibility 

The  foregoing  discussion  has  dwelled  for 
the  most  part  on  tailoring  of  requirements  by 
the  Government.  This  is  because  the  Govern- 
ment must  perform  the  initial  effort.  Govern- 
ment personnel  are  organizing  the  solicitation 
and  are  dealing  with  Government  require- 
ments documents.  In  addition,  the  Govern- 
ment must  have  the  requirements  organized  in 
such  a fashion  that  the  requirements  are 
amenable  to  the  tailoring  process.  After  initial 
tailoring  the  contractor  becomes  more  heavily 
involved.  Ideally,  the  solicitation  should  con- 
tain language  that  encourages  the  contractor 
to  propose  cost  effective  waivers  that  are  con- 
sistent with  program  objectives.  Such  encour- 
agement must  include  language  that  clearly 
protects  the  contractor  from  being  nonre- 
sponsive  in  the  event  that  he  proposes  such 
waivers  or  changes  to  requirements. 

Completion  of  the  Effort 

The  majority  of  tailoring  is  not  achieved 
until  Full  Scale  Development  is  nearly  com- 
plete. The  need  for  many  additional  devia- 
tions and  changes  does  not  become  apparent 
until  the  detail  design  is  nearly  complete  and 
the  program  documentation  is  near  comple- 
tion. This  final  tailoring  can  be  considerably 
facilitated  by  a good  initial  tailoring  that 
forms  a good  base  for  the  continuing  effort. 

Timing 

The  tailoring  effort  is  influenced  by  the 
timing  of  the  application  of  requirements. 
Premature  application  of  specifications  and 
standards  can  result  in  premature  efforts  to 
tailor  such  requirements.  This  situation  is 
wasteful  of  time  and  money.  Premature  tailor- 
ing will  result.  The  entire  effort  is  fruitless 
and  generally  results  in  having  to  be  done 
again  at  a more  appropriate  time  based  upon 
later  knowledge. 

Incentives 

A good  tailoring  job  is  the  mark  of  an  able 
acquisition  manager  and  is  its  own  reward. 
The  incentive  to  conscientiously  perform  a 
thorough  tailoring  may  well  be  the  receipt  of 


a contract  award  in  which  case  further  incen- 
tives are  unnecessary. 

Extent  of  Tailoring 

The  tailoring  described  here  is  intended  to 
achieve  complete  establishment  of  the  apph- 
cability  of  primary  Government  documents  at 
the  contract  application  level,  paragraph  by 
paragraph  where  necessary.  Reference  docu- 
ments are  of  less  consequence  than  the  pri- 
mary or  first  tier  documents  and  should  not 
be  made  applicable  unless  they  relate  to  a crit- 
ical component,  subsystem  or  related  task.  In 
such  a case,  the  referenced  document  should 
be  given  primary  document  status  and  tai- 
lored for  contract  apphcation.  Otherwise  the 
tailoring  of  referenced  documents  should  not 
be  necessary. 

CURRENT  STATUS 

Industry 

Industry  agrees  with  the  principles  of  tai- 
loring that  have  been  mentioned  here.  Indus- 
try knows  from  past  experience  that  nothing 
fruitful  will  occur  until  the  implementation  of 
such  principles  is  accomplished,  until  the 
process  is  made  a part  of  the  basic  regulations, 
and  until  the  principles  are  accepted  at  the 
working  level  and  the  process  is  made  a part 
of  regular  practice. 

Military  Procuring  Activities 

• United  States  Air  Force-Regulation 
AFR  800-3,  entitled,  “Engineering  of 
Defense  Systems”  and  dated  1 June 
1976,  gives  the  program  manager  the  au- 
thority and  responsibility  to  tailor  re- 
quirements. Air  Force  Systems  Com- 
mand Regulation  800-25,  entitled,  “Ap- 
plication of  Military  Specifications  and 
Standards  to  DOD  Documents,”  requires 
that  only  essential  specifications  and 
standards  be  used  and  that  they  be  tai- 
lored in  application.  The  Aeronautical 
Systems  Division  is  conducting  training 
courses  on  tailoring  of  requirements.  A 
contemporary  electronics  contract  has  a 
special  provision  that  encourages  the 
contractor  to  review  military  specifica- 
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tions  and  standards  for  the  specific  pur- 
pose of  recommending  exceptions. 

• United  States  Naval  Air  Systems  Com- 
mand-AIR-510F,  Systems  Criteria 
Branch,  has  for  many  years  successfully 
tailored  major  categories  of  requirements 
for  application  to  solicitations  and  con- 
tracts. 

• United  States  Army-Army  Letter 
AMCRD-EM  dated  13  May  1975  added 
the  responsibility  of  exercising  control  o- 
ver  application  of  specifications  and 
standards  to  the  existing  Data  Review 
Boards. 

• Department  of  Defense— Deputy  Secre- 
tary of  Defense  Clements’  4 August 
1975  letter  stated  that  specifications  and 
standards  should  not  be  invoked  without 
a specific  “scrub  and  tailor”  process.  De- 
partment of  Defense  Directive  4105.62 
requires  that  a review  board  thoroughly 
examine  requirements.  This  Directive 
4105.62  provides  for  feedback  from  con- 
tractors either  directly  or  through  an  in- 
dustry association  to  eliminate  unneces- 
sary requirements  from  solicitations. 
Armed  Services  Procurement  Regula- 


tions (ASPR)  Case  75-135  recommends 
changing  ASPR  Clause  1-1201  to  express 
minimum  needs  by  tailoring  the  applica- 
tion of  specifications  and  standards. 


FOR  THE  FUTURE 

The  Defense  Science  Board  Task  Force, 
sanctioned  by  the  Government,  has  recom- 
mended solution  of  this  problem  “by  an  im- 
mediate program  throughout  the  Department 
of  Defense  and  Industry  to  improve  the  cli- 
mate of  contractual  application”  and  states, 
“the  first  step  must  be  a joint  govemment/in- 
dustry  effort  to  effectively  tailor  the  contract- 
ual application  of  specifications  and  stand- 
ards.” The  Defense  Material  Specifications 
and  Standards  Office  (DMSSO)  of  the  Office 
of  Secretary  of  Defense  for  Installations  and 
Logistics  (I&L)  will  establish  projects  to  carry 
out  the  recommendations  of  the  Defense  Sci- 
ence Board.  The  Technical  Management  Com- 
mittee of  the  Aerospace  Industries  Associa- 
tion is  prepared  to  establish  comparable  proj- 
ects and  furnish  sponsors  to  interface  with 
DMSSO.  With  Government  and  Industry  dedi- 
cated to  the  successful  estabhshment  of  the 
tailoring  process,  much  progress  can  be  expec- 
ted in  the  foreseeable  future.  □ 


Mr.  William  F.  Brown  is  a 
program  coordinator  and  con- 
sultant. Mr.  Brown  retired 
from  the  Convair  Division  of 
General  Dynamics  Corpora- 
tion after  rendering  36  years 
of  service.  Primary  assign- 
ments at  Convair  included 
duty  as  Specifications  Group  Supervisor,  Chief  of  Data  Con- 
trol, Manager  of  Contract  Technical  Requirements  and  Con- 
vair Data  Manager,  and  Project  Coordinator. 


Mr.  Brown  was  the  General  Dynamics’  representative  to 
the  Technical  Management  Committee  of  the  Aerospace  In- 
dustries Association  and  is  past  chairman  of  that  Committee. 
This  article  is  based  upon  material  contained  in  a paper  that 
he  prepared  on  this  subject  at  the  request  of  the  AIA  Aero- 
space Technical  Council.  The  article  presented  here  reflects 
Mr.  Brown’s  experience  in  the  management  of  specifications, 
change  control,  data  management,  and  contract  technical  re- 
quirements. 

Mr.  Brown  attended  the  University  of  California  at  Los 
Angeles. 
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A System 
Management  Tool 


by 

Mr.  Eugene  Lurcott,  RCA 


A system  engineering  management  tool  that  uses  a proved  technique  is  described  in  this 
article.  The  technique  coordinates  the  interplay  between  system  engineers  and  engineering 
specialists,  and  simplifies  the  attainment  of  a balanced  system  design  in  which  each  major 
design  is  based  upon  the  proper  coordination  of  system  variables.  These  variables  include 
such  items  as  facilities,  equipment,  computer  programs,  personnel,  training,  testing,  and 
intrasystem  interfaces. 


INTRODUCTION 

he  process  described  in  this  document  as- 

A sists  in  the  definition,  control,  and  audit 
of  a balanced  system  design.  This  process  en- 
ables translating  the  operational  and  system 
requirements  into  specifications,  test  plans, 
and  procedures;  and  provides  the  backup 
data  required  to  define,  audit,  checkout,  and 
test  the  system. 

THE  TECHNIQUE 

The  Functional  Flow  Diagrams  and  De- 
scriptions,  F^D^,  define  and  interrelate  the 
functions  of  electronic  equipment,  mechani- 
cal equipment,  computer  programs,  facilities, 
and  personnel  that  constitute  a system.  The 
system  is  defined  in  user-imposed  require- 
ments, specifications,  or  other  baseline  docu- 
ments. The  approach  is  a “top-down” 

functional  analysis  of  the  system,  using  an 
increasingly  more  detailed  functional  break- 
down arranged  in  levels  or  “tiers.”  Each  tier 
is  a complete  functional  representation  of  the 
system  at  an  essentially  consistent  level  of 
definition.  The  implementation  of  the  tech- 
nique is  embodied  in  Functional  Flow  Dia- 


grams and  Descriptions  (F^D^)  and  in  Se- 
quence and  Timing  (SAT)  Diagrams. 

With  the  functional  definition  of  the  sys- 
tem complete,  the  functional  response  to  any 
scenario  within  the  scope  of  the  system  re- 
quirements becomes  a subset  of  that  defini- 
tion. Linking  the  functions  (at  any  one  level 
of  definition)  in  the  sequence  of  use  in  the 
scenario  results  in  what  are  called  Sequence 
and  Timing  (SAT)  Diagrams.  Because  the  SAT 
diagrams  arrange  the  functions  sequentially, 
the  cumulative  system  response  times  to  the 
selected  scenario  can  be  indicated.  Thus  the 
SAT  Diagrams  can  reveal  if  all  functions  are 
incorporated  into  the  system  to  allow  re- 
sponse to  a given  scenario  and  if  the  response 
times  are  within  requirements.  The  diagrams 
also  serve  in  the  development  of  system  test 
plans  and  procedures. 


f2d2  development 

The  fundamental  process  of  the  F^D^  sys- 
tems engineering  tool  is  shown  in  Figure  1. 
The  process  ties  into  Military  Standard  490 
which  sets  forth  practices  for  the  prepara- 


Vol.  I,  No.  4 


19 


System 
requirements 
translated  into 
functional  require- 
ments 


Tier  0 


Tier-0  functions  analyzed 
and  partitioned  into  require- 
ments for  subsystem  functions 


Tier  1 


Subsystem  functions  analyzed  and  parti- 
tioned into  requirements  for  major  equip- 
ments, manned  operating  stations,  and  computer 
programs. 


Tier  2 


Major  equipment,  manned  operating  station,  and  computer 
program  functions  analyzed  and  partitioned  into  requirements 
for:  (1)  cabinet  drawers  and  consoles,  (2)  operator  tasks,  (3) 
computer  program  modules. 


Tier  3 


Figure  1. 


The  Four  Tiers  of  the  Functional  Analysis  (F^D^)  Process  (a  top-down  approach  for 
total  system  definition). 


tion  and  interpretation  of  program-peculiar 
specifications  prepared  for  military  use. 

Tier-0 

The  first  step  in  the  process  is  initi- 

ated by  identifying  the  system  requirements. 
(See  Figure  2.)  These  requirements  may  be 
contained  in  top-level  work  statements, 
specifications,  or  other  baseline  documents. 
The  requirements  are  translated  into  func- 
tional requirements,  i.e.,  statements  of  op- 
eration. These  operation-oriented  require- 
ments are  presented  in  the  form  of  a top- 
level  (Tier-0)  functional  flow  diagram.  The 
number  of  Tier-0  functions  into  which  the 
system  is  divided  is  governed  by  functional 
complexity.  In  most  cases,  except  for  “In- 
put Interface”  and  “Output  Interface,”  the 
function  titles  are  self  explanatory.  All  in- 


puts to  the  system  come  in  through  “Input 
Interface”  and  all  outputs  from  the  system 
leave  through  “Output  Interface.”  This  pro- 
cedure aids  in  accounting  for  all  external 
interfaces  and  also  bounds  the  system.  As  in 
all  F^D^  diagrams,  inputs  come  in  to  the 
left,  and  outputs  leave  from  the  right.  At  the 
Tier-0  level,  data  paths  are  limited  to  basic, 
high-level  information  sufficient  to  show  how 
the  functions  tie  together. 

When  applied  to  definition  of  a Depart- 
ment of  Defense  procurement,  the  Tier  0 of 
the  F^D^  is  used  for  paragraph  3.1.4  “Sys- 
tem Diagrams”  of  the  Type-A  System  Speci- 
fication as  defined  in  MIL-STD-490.  This 
paragraph  incorporates  the  system-level  func- 
tional schematic  diagrams,  shows  the  top-level 
functional  flow  diagram  of  the  system,  and 
documents  the  abstractions  of  the  functions. 


20 


Defense  Systems  Management  Review 


Donahue,  Patrick  Shan 

10:56651 

ftDft04535£ 

DEFENSE  SYSTEMS  WNP6 
\KELLY,  ALBERT  J. 
due: 11/13/1997,23:59 


iisnd  .swilorjoil 

lcclSi^:QI 
iidt2f!Wuj4 
6A;i(!H  cMifeva  aeitoVa 
.1  THiiJl  ,yjjd/i/ 
0C;  Lfi  Jt't'l  \Li  \1  i ; sub 


EXTERNAL  TRACK 


OATA 

PLATFORM  STATUS 


EXTERNAL 

lOENTIFICATION 


MISSION  OROERS 


N 

c 

0 

M 

1 

N 

G 

I 

N 

T 

E 

R 

F 

A 

C 

E 


1 


O 

U 

T 

G 

0 
\ 

N 

G 

1 

N 

T 

E 

R 

F 

A 

C 

E 


assignments  to 

EXTERNAL  WEAPONS 


’ MISSION 
REPORTS 


Figure  2.  Example  of  a Tier-0  Functional  Flow  Diagram. 


Tier-1 

Tier  1 represents  the  second  step  of  the 
process.  In  this  step,  top-level  (Tier-0) 
functions  and  the  associated  criteria  are  anal- 
yzed and  translated  into  design  requirements 
to  be  allocated  to  specific  elements  (subsys- 
tems) of  the  overall  system.  At  this  point 
the  significant  characteristic  is  a functional 


definition  and  allocation  of  these  functions 
to  segments.  Figure  3 is  an  example  of  a Tier- 
1 diagram.  In  Figure  3 one  of  the  blocks  in 
the  Tier-0  diagram,  Figure  2,  is  detailed. 

In  general,  each  block  in  a Tier-0  F^D^  re- 
sults in  a separate  sheet  of  the  corresponding 
Tier-1  F^D^. 


4.  ACQUISITION  ANO  TRACK 


Figiu’e  3.  Example  of  a Tier-1  Diagram.  (The  “4”  in  the  title  refers  to  Block  4 in  the  preceding  diagram.) 
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Figure  4.  Example  of  a Tier-2  Diagram.  (The  “4.3”  refers  to  Block  4.3  in  the  preceding  diagram.) 


At  the  Tier-1  level,  implementation  of  the 
functions  is  not  addressed.  Thus  whether  the 
Tier-1  functions  are  to  be  performed  by  per- 
sonnel, by  equipment,  or  by  computer  pro- 
gram is  not  considered.  What  is  addressed  is 
the  division  of  the  Tier-0  functions  into  the 
Tier-1  functions  necessary  to  perform  the 
various  tasks  (e.g.,  data  acquisition,  analysis, 
decisionmaking,  and  control).  This  division 
into  functions  enables  defining  the  critical 
data  paths  and  the  interrelationship  between 
the  Tier-1  functions. 

Another  important  part  of  the  Tier-1  step 
is  the  systems  engineering  studies  which  are 
performed  concurrently  with  the  functional 
analysis  to:  (1)  select  from  alternate  choices 
of  function  sequences,  (2)  determine  the  best 
system  design  approach,  and  (3)  trade  off 
among  choices  of  element  allocations. 

Tier-2 

The  next  step  in  the  process  utilizes 

the  design  approach  determined  from  the  sys- 
tems engineering  studies  and  the  Tier-1  func- 
tional design.  The  element  functions  are  ana- 
lyzed in  sufficient  technical  detail  to  allow 
the  allocation  of  the  Tier-2  functions  to:  (1) 
equipment,  (2)  computer  programs,  or  (3) 
personnel.  Figure  4 is  an  example  of  a Tier-2 
diagram.  Note  that  one  of  the  blocks  shown 
in  Figure  3 is  detailed  in  Figure  4. 


The  Tier-2  step  also  includes  element 
studies  which  are  performed  concurrently 
with  the  functional  analysis  to:  (1)  determine 
the  design,  personnel,  training,  and  procedural 
data  requirements  imposed  by  the  function, 
and  (2)  select  the  optimum  design  approach 
for  integrating  the  design  requirements  into 
the  Prime  Item  Development  Specification 
(Bl)  and  the  Computer  Program  Performance 
Specification  (CPPS)  for  the  element. 

Tier-3 

In  those  areas  of  the  system  requiring  more 
detail  than  was  developed  in  Tier-2  analysis, 
a Tier-3  F^D^  may  be  required.  Based  on  the 
functional  analysis  and  element  engineering 
studies  performed  in  Tier  2,  the  functions  are 
analyzed  in  great  detail  and  are  translated 
into  operator-task  requirements,  hardware- 
design  requirements,  and  computer-program 
design  requirements.  Figure  5 is  an  example 
of  a Tier-3  diagram. 

Performed  concurrently  with  the  func- 
tional analysis  are  design  studies  aimed  at  op- 
timizing implementation  of  the  functions  al- 
located to  equipment,  computer  programs, 
and  operator  tasks.  These  design  studies  and 
analysis  result  in  the  support  of  full  defini- 
tions and  descriptions  of  the  functions  in  the 
appropriate  specification,  i.e.,  type  B-2  speci- 
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Figure  5.  Example  of  Tier-3  Diagram.  (The  “4.3,5”  refers  to  Block  4.3.5  in  the  preceding  diagram.) 


fication  for  critical  item  development  (per- 
formance) and  computer  program  perform- 
ance specifications.  The  studies  also  support 
non-specification  documents  such  as  man- 
machine  interface  documents. 

PREPARATION  OF 
FUNCTIONAL  FLOW 
DIAGRAMS 

Functional  Flow  Diagrams  are  prepared 
in  the  same  manner  as  other  engineering 
sketches  and  serve  to  translate  system  require- 
ments into  functional  terms.  Accuracy  and 
completeness  of  the  diagrams  are  more  im- 
portant than  format.  However,  to  avoid  mis- 
understanding, certain  conventions  and  sym- 
bols have  become  standard.  The  rules  and 
basic  symbology  for  the  diagrams  are  dis- 
cussed below. 

Functional  Definition  and  Numbering 

A function  is  defined  not  by  a name  but 
rather  by  the  input  data  upon  which  it  oper- 
ates and  by  the  output  it  produces.  All  func- 
tions are  represented  by  rectangular  boxes. 
These  boxes  may  be  visualized  as  “transfer 
functions”  in  which  input  data  are  operated 
upon  to  give  output  data. 


Functions  at  each  tier  are  numbered  in  a 
manner  that  preserves  the  continuity  of  the 
functions  and  enables  tracking  the  functions 
through  the  system.  Functions  at  a top  level 
(Tier  0)  are  numbered  1, 2,3,4,  etc.  Sub- 
functions of  these  top  level  functions  con- 
tain the  same  parent  identifier  and  are  coded 
at  the  next  decimal  level.  For  example,  the 
first  indentured  function  (Tier  1 ) of  function 
3 would  be  3.1,  the  second  indenture  (Tier 
2)  3.1 .1 , etc. 

For  expansion  of  a function  within  a par- 
ticular level  of  indenture,  a numerical  se- 
quence is  used.  For  example,  the  numerical 
sequence  used  to  amplify  function  3 at  the 
Tier-1  level  would  be  3.1 , 3.2,  3.3,  3.4,  etc. 

Experience  has  shown  that  each  tier  should 
amplify  the  definition  of  the  higher  level 
function  approximately  5 or  6 times.  How- 
ever this  will  vary  with  the  complexity  of  the 
function  being  amplified. 

A basic  ground  rule  of  F^D^  is  that  dif- 
ferent tiers  do  not  appear  on  a single  func- 
tional flow  diagram.  That  is,  the  output  of 
a Tier-1  function  is  not  the  input  of  a Tier-2 
function.  Tier-1  functions  talk  only  to  each 
other,  just  as  Tier-2  functions  talk  only  to 
each  other. 
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Tier  1 

ALLOCATION  SYMBOLOGY 

^ SPY-1 

ALLOCATION  TO  SYSTEM  OR  ELEMENT. 

Tier  2 

• 

COMPUTER  PROGRAM. 

A 

▲ 

MANNED  OPERATOR  STATION. 
EQUIPMENT. 

Tier  3 

• 

COMPUTER  PROGRAM. 

A 

■ 

EQUIPMENT. 

OPERATOR  DECISION. 

OPERATOR  ACTION 

DATA  SYMBOLOGY 

a 

♦ 

DATA  STORAGE  OR  FILE. 
CLOCK  OR  TIME  LAPSE. 

SUPPORT  DATA. 

V 

CRITERIA. 

Figure  6.  Symbology. 
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Data  Flow 

Lines  connecting  functions  indicate  the 
functional  flow  and  do  not  represent  either  a 
time  lapse  or  any  intermediate  activity.  Num- 
bers at  the  end  of  the  lines  identify  the  source 
or  destination  function  for  the  data.  If  pos- 
sible, input  data  enters  a function  at  the  left, 
and  output  data  exits  the  function  from  the 
right.  A true  functional  analysis  will  include 
the  feedback  involved  in  the  system  opera- 
tion. In  the  flow  diagrams  this  is  accom- 
plished by  looping  the  data  around  the  func- 
tions involved. 

The  functional  flow  across  interfaces  en- 
ables the  mutual  audit  of  both  functions  and 
interfaces,  thus  ensuring  that  both  are  com- 
plete, compatible,  and  in  accordance  with 
specified  requirements. 


SYMBOLOGY 

As  shown  in  Figure  6,  there  are  two  classes 
of  symbology  used  in  the  functional-flow 
diagrams.  One  class  is  used  to  indicate  the 
functional  allocation,  and  the  other  class  is 
used  to  represent  different  typ^s  of  data 
source  or  data  sinks. 

Allocation  Symbology 

Allocation  symbols  are  located  at  the  lower 
left  corner  of  the  function  box  and  are  used 
as  follows: 

Tier  1— Allocations  of  functions  at  the 
Tier-1  level  are  to  major  subsystems  or  ele- 
ments. These  allocations  are  shown  as  sub- 
system or  element  initials  in  an  ellipse. 

Tier  2— Allocations  of  functions  at  the 
Tier-2  level  are  to  computer  programs, 
manned  operator  stations  (an  operator  in 
conjunction  with  a console  or  panel),  or 
equipment.  A function  allocated  to  a com- 
puter program  is  represented  by  a solid 
circle.  A function  allocated  to  a manned 
operator  station  is  represented  by  a solid 
pentagon.  A function  allocated  to  equip- 
ment is  represented  by  a solid  isosceles 
triangle. 


Tier  3— Allocation  of  functions  at  the  Tier- 
3 level  is  to  specific  computer  program 
modules  or  algorithms,  operator  tasks,  or 
specific  equipments.  These  are  represented 
as  shown  in  Figure  6,  with  the  specific 
implementation  notated  as  operator  ini- 
tials, computer  program  label,  or  equip- 
ment name. 

Data  Symbology 

Data  symbology  is  also  shown  in  Figure  6. 
Frequently  data  generated  by  one  function  is 
used  at  a later  time  by  another  function.  This 
storage  of  data  is  shown  on  the  flow  diagrams 
by  a “file”  symbol.  These  “file”  symbols 
represent  the  complete  spectrum  of  types  of 
data  storage  required  in  a system,  ranging 
from  high  speed  (e.g.,  magnetic  core  mem- 
ories), to  very  low  speed  (e.g.,  grease  pencil 
on  a plot  board). 

The  performance  of  some  functions  is  con- 
trolled by  a clock  or  time  lapse.  This  time 
lapse  is  represented  symbolically  by  a dia- 
mond located  at  the  top  of  the  function. 

Many  functions  require  support  data  to  al- 
low the  function  to  perform.  Support  data  are 
defined  as  the  type  of  data  that  neither  initi- 
ates the  function  nor  dictates  how  the  func- 
tion operates.  Examples  of  support  data 
include  displayed  data,  ship  speed  and  head- 
ing, special  intelligence.  Support  data  usually 
vary  during  the  mission  and  are  portrayed  by 
a triangle  with  an  inscribed  “S”. 

The  last  type  of  data  used  by  a function  is 
criteria  type  information.  Criteria  data  are 
defined  as  that  data  that  establish  the  bound- 
ary conditions,  thresholds,  limits  of  opera- 
tion, priorities,  etc.  Criteria  data  define  the 
constraints  within  which  a function  can  per- 
form. Criteria  inputs  are  presented  by  closed 
triangles  at  the  top  of  the  function.  The  cri- 
teria are  identified  by  name  or  generically. 


FUNCTIONAL  DESCRIPTIONS 

The  Functional  Flow  Diagrams  may  be 
considered  as  system  “road  maps”  at  pro- 
gressively deeper  levels  of  detail.  Accompa- 
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FUNCTION  NAME;  Process  System  Tracks 
ABSTRACT; 


FUNCTION  NUMBER:  4.3 


This  function  initiates  and  maintains  automatic  system  track  processing. 
It  also  implements  manual  track. 


Incoming  Data 


Outgoing  Data 


New  System  Track  Data 
Updated  Track  Data 
Manual  Track  Request 


New  Track  Alert 
Smoothed  System  Tracks 
Video  Update  Request 


CRITERIA: 


This  function  implements  automatic  track  processing  by  operating  on  radar 
data  received  in  digital  format  and  implements  manual  track  by  operating 
on  radar  video. 

Automatic  track  processing  includes  initiation  of  the  track  file  in  the 
Central  Track  Stores  (CTS),  for  each  new  track  that  enters  the  system, 
generation  of  smoothed  position  and  velocity  data  from  update  data  and 
predicting  the  track  position  at  a subsequent  time.  Each  New  System 
Track  candidate  is  specially  correlated  with  tracks  in  the  CTS  to  avoid 
initiating  new  tracks  for  lost  tracks  reacquired  by  the  radar.  Reports 
which  represent  Reacquired  Lost  Tracks  are  processed  as  track  updates. 

Reports  which  do  not  specially  correlate  are  processed  as  New  System 
Tracks.  A System  Track  Number  (STN)  is  assigned  and  forwarded  to  the 
radar  and  the  track  smoothing  filter  (*C,^  filter)  is  initialized. 

Processing  of  each  track  update  report  begins  with  a check  for  presence 
of  a System  Track  Number  (STN).  Track  reports  that  do  not  include  an 

STN  occur  when  the  radar  has  updated  a track  before  it  associated  the  STN 

with  its  track  data.  Track  reports  that  include  an  STN  are  correlated 
with  the  CTS  track  data  via  STN;  track  reports  that  do  not  include  an  STN 

are  correlated  with  the  CTS  track  data  via  the  radar  track  number.  The 

processing  that  follows  correlation  includes  estimation  of  smoothed 
position  and  velocity  by  means  of  theo<,/&  filter,  predicting  position  of 
the  track  at  the  expected  track  update  time  and  forwarding  the  data  to 
the  Identification  Functions. 

Manual  track  is  initiated  by  decision  of  an  operator.  The  operator  ball- 
tabs  a location  on  a display  (near  which  he  has  reason  to  believe  a target 
of  interest  is  present),  and  requests  a video  update.  When  the  requested 
video  appears  on  the  display  he  ball -tabs  the  video  and  depresses  the 
"New  Track"  AEB.  This  initiates  manual  track  processing.  After  a reasonable 
time  lapse,  the  operator  requests  another  video  update.  When  the  requested 
video  appears  on  the  display  he  hooks  the  manual  track  and  depresses  the 
"Position  Correct"  AEB.  This  produces  processing  which  generates  smoothed 
position  and  velocity  and  the  predicted  position  at  the  next  update  (the 
next  expected  video  update  request).  This  process  is  repeated  until  the 
operator  decides  to  either  designate  the  track  for  automatic  processing  or 
to  drop  the  track. 

ALLOCATION: 

This  function  is  allocated  to  Command  and  Decision.  The  automatic  and 
manual  track  requirements  for  this  function  are  allocated  respectively 
to  paragraphs  3. 4. 1.2. 3 and  3. 4. 1.2. 4 of  the  WS  123456  specification. 

Figure  7.  Example  of  Functional  Description  which  Accompanies  each  Block  of  a Functional  Flow  Diagram. 
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nying  each  function  block  at  each  level  is 
text  material  giving  a functional  description 
of  the  particular  functions.  Figure  7 is  used  as 
follows. 

Abstract 

The  abstract  is  a concise  description  of  why 
the  function  is  in  the  system,  and  what  it 
does. 

Incoming  and  Outgoing  Data 

The  incoming  and  outgoing  signals  and/or 
other  data  is  listed  here  in  greater  detail 
than  can  readily  be  depicted  on  a flow  dia- 
gram. 

Criteria 

The  criteria  is  the  major  section  of  the 
functional  description.  How  the  function 
operates  on  the  inputs  to  obtain  the  out- 
puts, at  a level  of  detail  governed  by  the 
tier  level  of  the  particular  functions  is 
described  here.  If  required  for  complete 


understanding  of  the  operation  of  the 
function,  formulas  and  limits  are  included. 

Allocation 

This  is  an  extremely  important  section  that 
identifies  the  specification(s)  and/or  other 
design  document(s)  and  the  particular 
paragraph(s)  within  each  such  document 
that  specifies  the  function.  This  cross- 
reference  enables  a complete  system  audit 
by  making  all  functions  traceable  to  ap- 
proved documentation  and  all  documenta- 
tion traceable  to  allocated  functions. 

SEQUENCE  AND  TIMING 
DIAGRAMS 

A useful  adjunct  to  the  basic  is  the 

Sequence  and  Timing  (SAT)  Diagrams.  A SAT 
Diagram  is  a scenario-oriented  “needle  and 
thread”  pass  through  the  F^D^.  A SAT  Dia- 
gram uses  the  functions  tied  together  in  the 
same  way  as  in  F^D^,  but  with  only  the  in- 
puts and  outputs  applicable  at  that  function 
of  the  scenario. 


GUIDED  MISSILE 
LAUNCHING  SYSTEM 


UW5/ASROC 


FIRE  CONTROL  SYSTEM 


Figxu-e  8.  Part  of  a Sequence  and  Timing  Diagram  (SAT). 
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The  SAT  Diagram  links,  in  serial  form, 
the  functions  involved  in  any  particular  sys- 
tem usage  scenario.  Thus  the  diagram  is  help- 
ful in  predicating  the  response  time  of  the 
system  to  that  scenario.  Moreover  the  SAT 
diagram  is  helpful  in  system  test  planning  to 
ensure  that  all  system  functions  are  exercised. 
SAT  diagrams  tend  to  be  large;  Figure  8 is  a 
rudimentary  example. 

SUMMARY 

F2d2  is  a systems  engineering  manage- 
ment tool  that  provides  a complete  function 
definition  of  the  system  under  development. 
The  F^D^  translates  the  missions,  goals  and 
requirements  of  the  specifications  into  func- 
tional diagrams  and  functional  descriptions 


for  every  level  of  system  operation.  As  a tool 
for  system  definition,  F^D^  provides  the 
baseline  from  which  all  functions  are  quanti- 
fied and  allocated.  As  an  auditing  tool,  it 
provides  the  visibihty  required  to  ensure  that 
all  functions  have  been  incorporated  in  the 
design  and  that  the  design  is  in  accordance 
with  the  system  specification.  Design  control 
is  supported  through  the  combined  use  of 
definition,  audit,  and  the  associated  func- 
tional descriptions. 

Because  of  its  proved  value  in  the  develop- 
ment of  the  first  AEGIS  engineering  develop- 
ment model,  F^  D^  is  being  used  in  the  design 
and  development  of  the  AEGIS  Ship  Combat 
System.  In  addition,  F^D^  is  being  imposed 
on  programs  throughout  RCA.  □ 
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Observations 
on  Defense  Acquisition 

By 

Dr.  JosGph  F . ShGa,  RaythGon  Company 


Research  and  development  requires  an  effective  atmosphere  in  which  to  flourish.  Elements 
that  contribute  to  the  creation  of  useful  products  include: 

• Consistent  commitment  to  reasonably  defined  goals, 

• Balanced  design, 

• Technical  and  management  disciplines, 

• Recognition  of  uncertainty, 

• Product  follow  through  by  the  developer,  and 

• Experienced  people  and  well  defined  organizational  relations. 

In  this  article  the  author  provides  an  assessment  of  present  Department  of  Defense  (DOD) 
acquisition  practices,  as  related  to  these  elements,  with  a view  toward  possible  improvements. 


GOALS 

The  goal  of  those  charged  with  DOD  ac- 
quisition is  to  equip  the  US  Armed 
Forces  to  assure  a military  capability  superior 
to  that  of  any  potential  enemy.  But  such  a 
statement  is  too  general  to  be  useful.  Superi- 
ority must  be  translated  into  how  many  of 
what  things  are  to  be  developed  and  procured. 
Complex  issues,  ranging  from  the  present  and 
projected  capabilities  of  potential  enemies  to 
the  roles  of  the  individual  services,  are  explic- 
itly interwoven  with  the  process  of  setting 
requirements.  By  the  time  a specification  for 
a new  piece  of  defense  equipment  is  ready  for 
procurement  it  has  been  shaped  and  con- 
strained by  many  detailed  choices  involving 
mix  of  forces,  types  of  weapons  systems, 
mechanization  of  subsystems  and  availability 
of  technology. 

There  is  a danger  that  our  mathematical 
ability  to  analyze  and  simulate  can  create  the 


illusion  that  requirement  generation  is  a 
science.  The  fact  is  masked  that  as  require- 
ments generation  pits  conviction  against 
compromise,  and  quantity  against  perform- 
ance, it  is  a controversial  process  much  closer 
to  art. 

The  questions  at  all  levels  have  more  than 
one  possible  answer.  Study  team  after  study 
team  purports  to  derive  the  correct  solution 
where,  in  fact,  any  one  of  several  available 
options  could  be  equally  acceptable.  Inevit- 
ably, since  there  is  no  one  right  answer,  the 
decision  becomes  a function  of  the  experience 
and  personality  of  the  decisionmaker.  For- 
tunately, several  almost  equally  acceptable 
options  usually  exist,  so  the  problem  is  not 
one  of  making  “the  right”  decision.  Because 
of  the  ever  larger  number  of  people  with  di- 
verse background  who  are  involved-within  a 
service,  within  the  Department  of  Defense, 
and  increasingly  within  Congress— the  deci- 
sion process  has  become  very  long.  When  the 
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mean  time  to  decision  begins  to  approach 
the  tenure  of  the  decisionmakers,  instability 
or  paralysis  can  result. 

Consistent  commitment  to  reasonably  de- 
fined goals  requires  continuity  of  personnel 
and  a recognition  of  the  imprecise  nature  of 
initial  requirements.  The  implication  for 
defense  acquisition  is  a longer  tenure  for 
key  decisionmakers,  and  greater  flexibility 
to  adapt  requirements  to  the  reality  of 
development. 

The  results  possible  through  continuity  of 
key  personnel  and  organizational  structure 
can  be  dramatic.  Examples  include  the  con- 
sistent achievements,  over  more  than  2 
decades,  of  the  Fleet  Ballistic  Missile  Program 
and  the  Navy  Nuclear  Propulsion  Program. 
Other  areas  of  potentially  equal  importance, 
but  without  the  continuity  of  leadership  have 
made  little  progress  despite  the  fact  that  ap- 
plicable technology  is  not  a limitation.  To 
emphasize  the  point  still  further,  transforma- 
tion of  the  Russian  Navy  was  accomplished 
during  the  20-year  tenure  of  Admiral  Gorshkov 
as  Admiral  of  the  Fleet  of  the  Soviet  Union. 

Increased  performance  in  a weapons  system 
usually  means  a higher  price.  Within  the  con- 
straint of  DOD  budgets,  higher  price  per  unit 
translates  into  fewer  units.  Thus  the  articula- 
tion of  requirements  directly  impacts  the 
affordable  force  structure.  Historically  the 
problem  has  been  compounded  because  the 
requirements,  once  established,  were  consid- 
ered fixed.  Developments  complied-or  only 
belatedly  challenged  the  requirements  after 
protracted  problems  had  shown  the  require- 
ments to  be  impractical.  In  either  case,  the 
emphasis  on  performance  frequently  resulted 
in  higher  than  planned  production  costs 
forcing  reduced  operational  quantities. 

The  most  promising  antidote  for  this  his- 
toric malaise  lies  in  the  philosophic  principles 
behind  the  “design-to-price”  approach  to  con- 
tracting which  is  beginning  to  be  accepted 
within  Defense  Acquisition.  The  Government 
establishes  a unit  price  for  a new  equipment, 
along  with  a range  of  acceptable  performance. 
The  development  team  is  challenged  to  pro- 
vide the  highest  performance  (balanced  across 
a number  of  parameters)  for  the  stipulated 


price.  The  contractor  must  use  production 
cost,  in  addition  to  performance,  as  a major 
design  constraint,  and  the  Government  is 
forced  to  recognize  the  inevitable  interde- 
pendency of  performance  and  cost.  Effective 
implementation  of  design-to-price  can  be  a 
major  factor  in  checking  the  escalation  of 
defense  equipment  costs. 

BALANCED  DESIGN 

Research  and  Development  activities  exhibit 
two  characteristics  which  seem  to  be  almost 
universally  true: 

• The  output  of  a design  organization 
usually  improves  with  each  iteration. 
The  initial  design  cycle  rarely  approaches 
what  can  ultimately  be  achieved  in  per- 
formance, reliability  or  cost. 

• The  first  applications  of  new  technology 
are  always  accompanied  by  unexpected 
problems.  The  gains  in  performance,  size, 
weight,  or  cost  are  initially  bought  at  the 
price  of  development  delays,  perform- 
ance short  falls,  low  production  yields  or 
unexpected  operational  problems. 

A corollary  is  that  effective  development 
involves  evolutionary  improvements  in  prod- 
ucts by  experienced  design  teams.  New  tech- 
nology should  be  introduced  only  where  the 
payoff  is  significant.  In  other  words,  succes- 
sive generations  of  equipment  should  use 
what  has  been  proved  from  existing  design, 
and  stretch  the  “state  of  the  art”  only  where 
new  technology  is  mandatory  to  achieve 
the  goal. 

The  commercial  world  is  replete  with 
examples-the  telephone  system,  automobiles, 
appliances-where  the  successful  organizations 
steadily  evolve  their  product,  and  the  less 
competent  fall  by  the  wayside. 

How  does  DOD,  who  is  the  only  customer 
for  a wide  range  of  products,  handle  this 
issue? 

With  notable  exceptions,  continuity  is 
provided  more  within  the  Government  Lab- 
oratories than  within  contractor  organiza- 
tions. In  principle,  each  procurement  for  a 
new  generation  of  equipment  stands  on  its 
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own,  equally  available  to  all  comers  since 
“all  contractors  are  created  equal.”  Too 
often,  the  assumption  seems  to  be  that  pro- 
posals alone  provide  a sufficient  data  base 
to  accurately  select  the  best  qualified  con- 
tractor. The  fallacy  in  that  assumption  can 
be  judged  by  observing  the  number  of  pro- 
curements in  which  technical  normalization  is 
forced  on  the  bidders,  until  the  competition 
ultimately  becomes  a price  auction.  When 
this  happens,  particularly  with  cost  type  pro- 
curements, the  experienced  contractor  is  at  a 
disadvantage,  since  he  knows  both  the  prod- 
uct and  the  customer  in  detail.  When  cost 
becomes  the  major  evaluation  criterion, 
realistic  bidding  is  discouraged,  and  the  door 
is  opened  to  either  deliberate  buy  in  or 
inexperience. 

Perhaps  of  even  more  concern  is  the  ques- 
tion of  what  influences  the  procurement 
specifications.  The  Government  in-house  lab- 
oratories are  technically  oriented  and  tend  to 
display  the  proclivity  of  most  engineering 
organizations-an  attraction  toward  new 
technology.  This  trend  is  exacerbated  by 
contractors  who  attempt  to  create  competi- 
tive advantage  by  convincing  the  preparers  of 
the  Request  for  Proposal  that  the  fruits  of 
their  advanced  development  programs  are 
ready  to  be  plucked.  Individual  contractors 
usually  emphasize  different  aspects  of  a sys- 
tem, and  it  is  all  too  easy  for  resultant  speci- 
fications to  reflect  a spectrum  of  technology 
which  no  one  contractor  can  comfortably  en- 
compass, and  which  is  not  really  required  for 
the  mission. 

There  is  an  old  proverb  which  can  be  traced 
back  through  Voltaire  to  the  Greeks.  “The 
better  is  the  enemy  of  the  good.”  Too  often 
we  obsolete  an  existing  “good”  for  a pre- 
sumed “better”  only  to  find  the  benefits 
illusory  and  the  cost  exorbitant.  A new  sys- 
tem does  not  have  to  be  composed  of  all  new 
subsystems.  Aircraft  can  use  existing  com- 
munication or  guidance  systems,  radars  can 
use  existing  displays-the  list  can  be  quite 
long.  At  lower  levels,  the  use  of  standard  parts 
that  have  a production  pedigree  and  are  under- 
stood by  designers  can  lessen  development 
risk,  while  at  the  same  time  simplifying  future 
logistics. 


A few  segments  of  the  Defense  industry  do 
operate  in  this  fashion.  The  development  of 
aircraft  engines  has  devolved  to  only  two  com- 
panies and  new  developments  are  the  result  of 
carefully  nurtured  product  improvement  and 
advanced  technology  progranis.  But  in  many 
sectors  the  Defense  industry  is  over  expanded. 
The  emphasis  is  on  technical  innovation  as  a 
road  to  survival.  Jacques  Gansler’s  study  of 
the  DOD  Industrial  Base  shows  both  over 
capacity  at  the  system  level,  and  atrophy  of 
capability  at  critical  subsystem  or  component 
levels. 

• 

Although  greater  emphasis  on  relevant  ex- 
perience in  procurement  might  be  viewed  as 
potentially  limiting  to  competition,  in  reality 
it  would  bring  DOD  much  closer  to  commer- 
cial practices  in  which  the  re-procurement 
actions  of  satisfied  customers  are  a major 
factor  in  the  free  enterprise  “survival  of  the 
fittest.” 

The  present  situation  in  Defense  Acquisi- 
tion may  be  the  result  of  a failure  of  presum- 
ably free  enterprise  policies  (competition 
open  to  all)  in  a monopsonistic  environment 
which  is  unable  to  generate  clear  cut  criteria 
to  select  superior  products.  Indeed,  the  excess 
of  capacity  in  some  areas  and  the  dearth  in 
others  may  eventually  dictate  more  con- 
sciously planned  management  of  the  indus- 
trial base  than  DOD  has  practiced  in  the  past. 

DISCIPLINE 

One  of  the  greatest  pitfalls  in  Defense 
Acquisition  is  a combination  of  inflexibility 
and  overkill.  Research  and  development  man- 
agement requires  discipline  across  a spectrum 
extending  from  detailed  design  through  cost 
and  schedule  control.  The  DOD  procurement 
structure  is  interlaced  with  requirements  for 
formal  procedures  designed  to  achieve  such 
discipline.  These  requirements  have  been 
mandated  by  successive  administrations  that 
tend  to  view  the  recurring  acquisition  prob- 
lems—overruns,  schedule  slippage,  costly  and 
unreliable  hardware— from  somewhat  differ- 
ent perspectives.  There  is  a tendency  to  gen- 
eralize a DOD-wide  solution  from  the  specific 
experience  on  a given  program.  Often  a new 
set  of  requirements  is  introduced  without 
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modifying  existing  practices.  The  result  is  a 
redundant  set  of  checks  and  balances  so  en- 
crusted with  refinements  that  the  rigor  sought 
comes  closer  to  rigor  mortis. 

Take,  for  example,  the  introduction  of 
Independent  Operational  Test  and  Evaluation 
as  a major  check  on  the  development  process. 
This  check  represents  a significant  and  posi- 
tive step-a  step  needed  because,  despite  the 
apparent  rigor  of  qualification  testing,  reli- 
ability testing,  maintainability  testing,  et  al, 
the  resultant  operational  hardware  too  often 
has  been  unsatisfactory.  In  addition  to  inde- 
pendent evaluation,  operational  testing  pro- 
vides a realistic  environment  for  the  collection 
of  reliability  and  maintainability  data.  Yet 
programs  continue  to  incur  the  delay  and 
attendant  cost  of  dedicated  reliability  and 
maintainability  tests  in  series  with  Indepen- 
pendent  Operational  Test  and  Evaluation. 

Might  it  not  make  more  sense  to  consider 
all  tests  relevant  to  the  assessment  of  the 
reliability  potential  of  new  equipment?  In 
particular,  the  logs  from  both  developmental 
tests  and  Independent  Operational  Test  and 
Evaluation  should  provide  sufficient  data  to 
assess  the  mean-time-between-failure  and  the 
mean-time-to-repair  of  the  equipment  under 
test.  In  addition  to  saving  time  and  money, 
the  test  conditions  would  be  more  realistic 
than  present  practices. 

Effective  checks  and  balances  should  stress 
content,  not  form.  But  the  continued  refine- 
ment of  a discipline  can  often  lose  sight  of  the 
original  intent.  Again,  consider  reliability . The 
elements  that  produce  a reliable  product  are 
conservative  design,  consistent  derating,  and 
pedigreed  parts  coupled  with  a rigorous  test 
program  that,  in  the  limit,  identifies  and  fixes 
the  cause  of  every  failure.  When  failure  follow 
up  is  continued  through  initial  field  experi- 
ence, reliability  can  be  expected  to  improve 
over  the  early  years  of  product  life. 

Most  reliability  programs  stress  independ- 
ent review  of  design  for  reliability  related 
matters  and  the  use  of  high  reliability  parts. 
The  programs  also  require  a relatively  long 
test  program,  conducted  under  somewhat  arti- 
ficial conditions,  that  is  designed  to  assess 


(with  statistical  significance)  how  close  the 
hardware  comes  to  meeting  its  mean-time- 
between-failure  specification.  The  tests  are 
usually  carried  out  on  developmental  hard- 
ware that  may  earlier  have  been  used  for 
qualification  testing,  or  on  early  production 
units  that  may  not  be  representative  of  the 
operational  population. 

Although  mathematically  elegant,  statisti- 
cal significance  is  probably  the  least  signifi- 
cant element  in  achieving  reliability.  In  the 
Apollo  Program,  usually  credited  with  being 
the  acme  of  reliability,  it  was  concluded  (after 
the  conventional  approach  to  reliability  had 
been  followed  for  a few  years)  that  statistical 
reliability  assessment  was  both  impractical 
and  unnecessary.  Rigorous  ground  testing  and 
failure  follow  up  provided  the  technical  and 
management  confidence  to  man  rate  the 
Saturn  V booster  after  only  two  unmanned 
launches;  one  to  be  sure  that  the  flight  regime 
was  as  expected,  and  the  second  to  be  sure 
the  first  was  not  a random  success. 

While  it  is  true  that  quantities  in  DOD  pro- 
curements are  larger  and  therefore  more  sus- 
ceptible to  statistical  analysis,  the  real  relia- 
bility data  lies  in  the  failures  encountered  in 
development  and  operational  test  programs, 
and  in  the  experience  with  deployed  field 
equipment.  Resources  programmed  primarily 
for  evaluation  might  better  be  spent  in  failure 
follow  up  and  corrective  action-a  positive 
investment  in  reliability  growth. 

Department  of  Defense  programs  range 
from  spacecraft  that  must  operate  untended 
for  years,  to  one  of  a kind  radar  systems 
manned  24  hours  a day;  from  missiles  that 
have  to  work  on  command  after  years  of  stor- 
age to  redundant  communications  equipment 
readily  repaired  with  replaceable  modules. 

Practices  appropriate  to  the  more  demand- 
ing applications  tend  to  spread  across  the 
board,  resulting  in  overkill  and  unnecessary 
cost.  For  example,  high  reliability  electronic 
parts  are  obviously  required  for  the  unmanned 
spacecraft.  Since  the  basic  chips,  and  in  most 
cases  the  packaging  and  assembly,  are  identi- 
cal, the  difference  between  high  reliability 
and  commercial  parts  these  days  lies  primarily 
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in  the  screening  and  burn-in  done  to  eliminate 
the  imperfect  devices  that  slip  through  the 
manufacturing  process.  Burn-in  can  reduce 
the  already  low  commercial  failure  rate  by  a 
factor  of  ten  or  so,  but  increases  the  price  by 
an  even  larger  factor.  The  cost  of  high  relia- 
bility parts  should  be  compared  to  other  ways 
of  achieving  satisfactory  operational  per- 
formance. For  example,  an  attended,  ground 
based  radar  could  be  assembled  with  commer- 
cial parts.  The  system,  in  effect,  would  do  its 
own  screening  in  the  operational  environment 
during  the  initial  months  of  operation.  Main- 
tenance is  required  in  either  case,  and  the 
slightly  higher  cost  of  replacing  one  or  two 
per  cent  of  the  devices  in  the  system  during 
the  burn-in  phase  (probably  with  high  relia- 
bility equivalents  from  Defense  Supply 
Agency  stores)  would  be  more  than  offset 
by  the  lower  acquisition  cost.  Incidentally, 
the  increased  failure  rate  might  even  provide 
the  better  training  experience  for  a mainte- 
nance crew. 

In  the  absence  of  concentrated  manage- 
ment attention,  opportunities  for  effective 
implementation  of  requirements  can  be  in- 
hibited by  the  conservatism  inherent  in  an 
organization  as  large  and  structured  as  DOD. 
Military  Specifications  and  Standards  are  a 
case  in  point.  In  a recent  study*  it  was  found 
that  these  documents,  long  a target  for  criti- 
cism, generally  contain  much  more  flexibility 
than  appears  to  be  used  in  practice.  Industry 
was  as  guilty  of  over  interpretation  as  Gov- 
ernment was  of  over  enforcement.  Greater 
payoff  can  be  expected  from  changing  the 
method  of  application  of  specifications  and 
standards  than  from  improvement  in  the 
substantive  content.  The  Government  must 
recognize  the  inherently  arbitrary  nature  of 
standardization,  and  establish  a climate  where 
the  applicability  of  specific  provisions  can  be 
challenged  both  by  tailoring  specifications  to 
the  particular  needs  of  a given  program  and 
by  granting  cost  effective  waivers. 

Since  the  existing  procurement  environ- 
ment is  basically  conservative  and  encourages 
cautious  conformance  rather  than  forceful 


♦Defense  Science  Board,  “Report  of  the  Task  Force  on 
Specifications  and  Standards,”  15  Jan  77. 


ingenuity,  the  Government  Program  Manager 
and  the  organization  that  supports  him  must 
be  educated  and  motivated  to  realize  that 
strict,  parochial  application  of  specifications 
and  standards  is  neither  required  nor  desired. 

In  effect,  this  recommendation  to  empha- 
size content,  not  form,  is  a natural  extension 
of  design-to-price  principles  to  levels  of  detail 
not  normally  challenged. 

UNCERTAINTY 

The  cocoon  of  formality  that  encases  the 
development  process  .from  requirements, 
through  proposal  and  negotiation  to  contract 
award  often  hides  or  deliberately  subverts 
the  basic  uncertainties  of  maturing  new  equip- 
ment. Every  development  program  contains 
uncertainty,  since  at  least  part  of  the  equip- 
ment will  see  the  light  of  day  for  the  first 
time.  A new  design  can  not  be  brought  to 
fruition  without  encountering  some  problems 
along  the  way.  The  amount  of  trouble  is  a 
function  of  the  difficulty  of  design,  but  some 
unforeseen  problems  are  inevitable  even  in  the 
simplest  of  products.  People  are  fallible,  na- 
ture is  a harsh  task  mistress,  and  no  one  organ- 
ization can  amass  the  talent  (or  afford  to  con- 
centrate it  in  a single  area),  required  to  do 
things  perfectly.  Good  R&D  management  is 
the  art  of  controlling  uncertainty  with  accept- 
able precision. 

The  obvious  response  to  uncertainty  is  con- 
tingency, the  elbow  room  in  performance, 
schedule  and  cost  required  to  cope  with  the 
unexpected  while  maintaining  overall  program 
commitments.  Although,  as  noted,  design-to- 
price  has  introduced  a long  overdue  recognition 
of  the  flexibility  required  to  balance  per- 
formance and  production  cost;  development 
schedules  and  funding  remain  problem  areas. 

Programs  too  often  are  squeezed  into  an 
arbitrary  time  frame,  constrained  by  a re- 
quired initial  operation  capability  (IOC)  date 
at  one  end,  and  a contract  go-ahead  on  the 
other. 

Initial  planning  often  tends  to  be  realistic, 
with  adequate  blocks  of  time  for  develop- 
ment, test,  production  and  deployment.  But 
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when  the  in-house  Government  decision  proc- 
ess stretches  out,  there  is  reluctance  to  change 
the  IOC  date.  Pressure  is  put  on  contractors 
to  shorten  the  cycle,  or,  to  be  more  accurate, 
to  say  that  they  can  shorten  the  cycle.  Often 
the  net  effect  is  counter  productive.  Produc- 
tion lead  times  are  hard  to  change.  Therefore, 
development  and  test  tend  to  be  squeezed. 
Engineering  release  is  advanced,  shortening 
the  design  and  breadboard  effort,  and  test 
time  is  reduced,  lessening  the  ability  to  cope 
with  failures  encountered.  The  program  be- 
comes success  oriented  at  the  sacrifice  of  the 
essential  ingredients  of  success,  i.e.,  adequate 
design,  test  and  rework  time.  Program  slippage 
and  cost  overrun  become  almost  inevitable. 

An  obvious  solution  is  realistic  scheduling 
with  contingency  time  in  the  plan.  But  real- 
ism before  the  fact  is  in  the  eye  of  the  be- 
holder. Program  Managers  who  have  not 
grown  up  in  research  and  development  have 
little  experience  on  which  to  base  such  judg- 
ments; and  even  the  experienced  Program 
Managers  occasionally  become  cynical  or  suc- 
cumb to  the  presumed  pressures  of  the  system. 

The  funding  situation  can  be  equally  diffi- 
cult. Any  monies  identified  as  contingency 
funds  in  a contractor’s  proposal  are  usually  re- 
moved in  negotiation,  without  compensating 
provision  made  by  the  Government. 

Identified  contingency  provisions,  based  on 
the  degree  of  difficulty  of  a program,  could 
be  a major  step  in  schedule  and  cost  overrun 
reduction.  Whether  the  funds  are  retained  by 
the  Government  or  negotiated  into  contracts, 
and  how  such  resources  might  be  pooled  (sta- 
tistically) among  programs,  are  subsidiary 
issues  capable  of  a variety  of  solutions. 

PRODUCT  FOLLOW  THROUGH 

A qualification  test  program  or  Operational 
Test  and  Evaluation  cannot  be  comprehensive 
enough  to  uncover  all  possible  problems  that 
may  arise  when  complex  new  equipment  is 
put  into  operational  use.  Prompt  attention  to 
such  problems  can  be  a major  factor  in  achiev- 
ing and  even  bettering  reliability  and  main- 
tainability goals. 


In  the  commercial  sector,  the  developer  is 
usually  responsible  for  installation  and  main- 
tenance of  his  equipment  on  the  customer’s 
premises.  To  satisfy  a customer  the  mainte- 
nance must  be  timely  and  effective.  Good 
commercial  organizations  use  the  mainte- 
nance program  to  monitor  the  experienced 
mean-time-between-failure  and  the  mean-time- 
to-repair  of  the  fielded  hardware.  Anomalies 
can  be  quickly  identified,  and  corrective  ac- 
tion taken.  Redesign,  improved  training  or 
revised  manuals  can  be  expected  to  upgrade 
performance  during  the  early  years  of  product 
life. 

The  DOD  procurement  practices  too  often 
preclude  the  developer  from  following  his 
product  into  the  field.  Maintenance  is  per- 
formed by  uniform  personnel  supported  by 
depots.  Record  keeping  is  difficult.  Feedback 
to  the  developer  is  inadequate  or  nonexistent. 

Much  more  can  and  should  be  done  to 
keep  the  developer  in  the  loop  during  the 
early  years  of  product  life.  This  organization 
will  have  both  the  interest  and  the  expertise 
to  solve  the  problems  that  arise.  Based  on 
experience  with  commercial  operations,  such 
involvement  could  be  a major  factor  in  elimi- 
nating unsatisfactory  products  from  DOD 
inventories. 

PERSONNEL  AND 
ORGANIZATION 

Research  and  development  management  is 
a skill  that  must  be  learned  as  any  other  skill 
is  learned,  through  practice  tempered  by  per- 
formance. Research  and  development  manage- 
ment is  a career  of  fundamental  importance 
to  the  Services— a career  equivalent  to  that  of 
line  command  although  demanding  different, 
and  specialized,  training. 

Despite  the  apparent  precision  of  manage- 
ment systems  that  provide  data  and  procedures 
to  permit  anyone  with  the  right  credentials 
to  function  in  a given  slot,  in  the  real  world  it 
is  the  judgment  and  competence  of  the  indi- 
viduals involved  that  determine  how  well  a 
program  is  run. 
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In  a field  as  strewn  with  potential  pitfalls  as 
Defense  Procurement,  that  involves  a broad 
spectrum  of  technology  (contractual  require- 
ments and  organizational  complexity),  judg- 
ment can  be  gained  only  by  experience.  Indi- 
viduals can  be  seasoned  by  serving  at  several 
levels  of  responsibility,  learning  from  both  the 
successes  and  failures  found  along  the  way. 

Perhaps  the  best  initial  experience  for  a 
future  Program  Manager  is  to  work  on  a pro- 
gram from  inception  to  production,  or  devel- 
opment through  deployment.  It  is  the  only 
way  to  learn,  first  hand, 

• the  changes  that  take  place  as  system 
concepts  are  translated  into  designs, 

• the  lead  times  required  to  fabricate 
hardware, 

• the  nature  of  problems  encountered  in 
test, 

• the  problems  that  could  have  been  antic- 
ipated and  those  that  came  as  a surprise, 

• the  problems  of  production  start  up  . . . 
on  and  on. 

A Program  Manager  who  has  been  through 
the  mill  knows  when  a schedule  is  realistic, 
can  sense  how  much  contingency  is  justified, 
and  has  a feel  for  the  right  ballpark  on  cost. 
Of  even  more  importance,  he  can  have  the 
confidence  and  stature  to  do  what  is  right  for 
his  program,  to  innovate  in  the  interest  of 
economy,  and  withstand  pressures  from  out- 
side sources  that  attempt  to  impose  unreason- 
able requirements. 

You  learn  from  the  problems,  from  the  de- 
signs that  do  not  meet  specifications,  from 
the  missile  that  aborts,  from  the  gun  that 
hangs  fire,  from  overruns  and  schedule  slips, 
and  from  contractor  promises  vs  observed 
performance. 

There  is  no  one  correct  way  to  run  a pro- 
gram. Ultimately,  the  program  must  reflect 
the  personality  and  competence  of  the  Pro- 
gram Manager.  He  needs  to  be  given  some  lati- 
tude to  do  things  his  way,  especially  when  his 
track  record  is  proven. 

Acquisition  management  must  be  recog- 
nized as  a career  path  in  all  the  Services  to 


assure  the  availability  of  experienced  per- 
sonnel to  meet  DOD  needs. 

In  addition  to  experience,  a Program  Man- 
ager requires  effective  support  to  do  an  accept- 
able or  superior  job.  A competent  organization 
and  appropriate  delegation  of  authority  are 
required,  buttressed  by  effective  management 
support. 

The  Program  Office  is  the  interface  between 
government  and  industry,  the  major  check 
and  balance  on  contractor  performance.  The 
Program  Office  job  is  not  just  routine  evalua- 
tion of  cost  and  schedule  progress. 

Far  more  important  is  the  technical  assess- 
ment of  the  contractor’s  approach  and  prog- 
ress. The  government  experience  may  be 
broader  than  that  of  any  individual  contrac- 
tor. Technical  interchange  between  govern- 
ment and  contractor  is  essential  throughout 
the  life  of  a development  program. 

To  conduct  a development  program  effec- 
tively the  government  technical  people  who 
support  the  Program  Manager  must  be  moti- 
vated not  just  toward  technical  excellence, 
but  toward  a balanced  view  of  performance, 
cost  and  schedule.  Few  programs  can  afford 
the  “world’s  best”  anything.  Programs  will  be 
more  than  successful  if  they  can  approach  a 
“most  adequate”  status  across  the  board, 
where  most  adequate  is  defined  as  “a  little 
more  than  enough,  but  not  too  much.”  When 
the  technical  support  to  a Program  Manager 
comes  from  in-house  laboratories,  particu- 
larly those  geographically  and  organizationally 
separated  from  the  Program  Office,  proper 
motivation  may  be  difficult.  Yet  the  Program 
Manager  is  often  a captive  of  the  judgments 
exercised  by  his  technical  people  who  are 
almost  always  performance  driven.  This  aspect 
of  the  defense  acquisition  culture  is  perhaps 
the  most  difficult  to  change.  One  possible 
solution  is  to  emphasize  that  laboratory  sup- 
port assignments  are  delegations  of  subsystem 
responsibility— not  just  for  the  areas  of  techni- 
cal discipline  involved,  but  for  cost  and  sched- 
ule performance  as  well. 

The  Program  Manager’s  job  should  be 
downward  oriented,  working  with  his  organi- 
zation, the  contractors  and  the  test  and  using 
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Commands.  Reasonable  management  report- 
ing is  required.  But  too  often,  a typical  Pro- 
gram Manager  has  the  tremendous  additional 
burden,  especially  on  large  programs,  of  being 
his  own  advocate,  over  and  over,  defending 
against  reprogramming,  preparing  for  DSARCs 
and  rehearsing  for  intermediate  level  of  com- 
mand presentations.  He  lives  in  an  environ- 
ment that  is  careless  of  his  time,  distracts 
him  from  his  main  job  and  is  potentially 
frustrating. 


The  Program  Manager’s  job  is  not  easy.  The 
requirements  for  advocacy  for  his  program, 
both  within  and  outside  DOD  can  consume  a 
large  portion  of  his  time,  at  the  expense  of 
attention  to  the  main  task  of  running  the 
program.  Much  can  be  done  to  provide  the 
Program  Manager  stronger  support  and 
strengthen  his  ability  to  be  an  effective  deci- 
sionmaker for  his  program.  □ 


Dr.  J oseph  F.  Shea,  Senior 
Vice  President  and  Group 
Executive,  Raytheon  Com- 
pany, Lexington,  MA,  is  a 
former  member  of  the  Board 
of  Visitors,  Defense  Systems 
Management  College.  He  re- 
ceived a B.S.,  Mathematics, 
from  the  University  of  Michigan  followed  by  a M.S.,  Engi- 
neering Mechanics  and  a PhD,  Engineering. 

A former  instructor  of  Engineering  Mechanics  at  the 
University  of  Michigan,  Dr.  Shea  has  had  extensive  experi- 


ence in  positions  of  responsibility  in  industry  having  worked 
for  Bell  Telephone  Laboratories,  General  Motors  Corpora- 
tion, Space  Technology  Laboratories  and  Polaroid  Corpora- 
tion. For  a 4-year  period  Dr.  Shea  was  employed  at  the 
Manned  Spacecraft  Center,  NASA. 

Dr.  Shea,  a former  US  Navy  Reserve  Officer,  has  served 
as  a consultant  to  the  Secretary  of  Defense  and  chaired  the 
Task  Force  on  “Specifications  and  Standards,”  (1974-75). 
He  is  a Fellow,  American  Astronautics  Society;  Fellow, 
American  Institute  of  Aeronautics  and  Astronautics;  and 
is  a Member  of  the  ASME  and  the  IEEE. 
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Zero-Base  Budgeting 

and 

Sunset  Legislation 

by 

Lt.  Col.  Paul  B.  Demitriades,  USAF  Reserve 

Zero-base  budgeting  (ZBB),  an  innovative  budget  technique,  was  developed  in  1968  by  Peter 
A.  Pyhrr,  at  Texas  Instruments,  Inc.  In  1971,  Jimmy  Carter,  then  Governor  of  Georgia,  with 
Mr.  Pyhrr  as  an  advisor,  introduced  the  zero-base  budget  concept  for  preparation  of  the 
state’s  fiscal  year  1973  budget. 


Mr.  Phyrr’s  book,  Zero-Base  Budgeting:  A Practical  Management  Tool  for  Evaluating  Ex- 
penses,'^ is  the  first  manual  on  budgets  and  planning,  according  to  the  Washington  Post,  to 
have  caused  a run  on  Washington,  DC  bookstores.  The  current  popularity  of  this  manual  can 
be  traced  to  President  Carter’s  campaign  promise  that  “Immediately  after  my  inauguration, 
I will  require  zero-base  budgeting  for  all  Federal  Departments,  Bureaus,  and  Boards  by  Ex- 
ecutive  Order.” 


WHAT  IS  ZERO-BASE 
BUDGETING  (ZBB)? 

Most  Government  budgets  traditionally 
have  been  prepared  by  line-item  on  an 
incremental  basis,  and  allocation  decisions 
have  been  made  on  the  basis  of  prior  levels  of 
expenditure.  Thus  the  budget  for  any  given 
year  becomes  a product  of  the  previous  year’s 
budget.  Consequently,  programs  become  en- 
trenched, resistant  to  measures  of  program 
effectiveness  and  rational  decisionmaking. 

In  contrast  to  the  traditional  “numbers 
oriented”  approach  of  incrementing  the  new 
on  old  line  items,  “decision-oriented”  zero- 
base  budgeting  requires  that  each  program  or 
function— new  or  old— in  an  organization  must 
be  justified  in  its  entirety  each  time  a new 
budget  is  formulated.  The  method  does  not 
assume  any  increase  over  the  established  base 
of  expenditures,  hence  “zero-base.” 


There  are  four  basic  steps  to  zero-base  bud- 
geting: 

(1)  Identify  decision  units; 

(2)  Develop  program  decision  packages. 
This  requires  analyzing  and  describ- 
ing each  discrete  activity,  current  as 
well  as  new,  in  one  or  more  decision 
packages; 

(3)  Rank  program  decision  packages, 
(this  involves  evaluating  and  assign- 
ing a priority  to  the  packages, 
through  cost-benefit  analysis  and/or 
subjective  evaluation).  The  conse- 
quences/impact of  any  changes,  ei- 
ther from  the  current  or  recommen- 
ded budget  levels,  are  also  identified; 

(4)  Allocate  the  budget  on  the  basis  of 
information  derived  from  steps  1 
through  3. 

To  provide  decisionmaking  alternatives 
zero-base  budgeting  usually  requires  that  the 
budget  request  be  based  on  program  decision 
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packages  developed  to  support  several  lev^s 
of  operation,  as  illustrated  in  Figure  1 : 


Program  Decision  Packages 
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Figure  1 , Alternative  Program  Decision  Packages 

The  zero-base  budget  process  is  a bottoms- 
up  management  approach  to  planning  that  re- 
quires the  involvement  of  managers  at  all  lev- 
els of  organization.  The  process  demands  an 
analytical  framework,  is  usually  a revelation 
to  all  who  participate,  and  results  in  an  im- 
proved quality  of  subsequent  management 
judgments.**  Although  most  applications  of 
zero-base  budgeting  emphasize  cost  reduction, 
the  process  also  provides  an  increased  capabil- 
ity to  respond  quickly  to  growth  situations, 
since  all  activities  have  been  preplanned  and 
given  a priority.  Figure  2 illustrates  an  exam- 
ple of  format  and  content  of  a typical  pro- 
gram decision  package. ^ 


• Purpose/Objective 

• Description  of  action 

• Cost  and  benefits 

• Workload  and  performance  measures 

• Alternative  means  of  accomplishing 
objectives 


Figure  2,  Program  Decision  Package 


*For  a discussion  of  the  program  decision  package  for- 
mat, see  “State  of  Georgia,  Zero-Base  Budget  Procedures 
and  Instructions  for  Fiscal  Year  1978  Budget  Development,’ 
Office  of  Planning  & Budget,  June  1976. 

**  For  a review  of  ZBB  results  at  a Canadian  University  and 
several  utilities  see:  John  F.  MacFarlane,  “Zero-Base  Budget- 
ing in  Action,”  CA  (Chartered  Accountant),  XXV(6):  20 
(1976);  and,  Paul  J.  Stonich  and  William  H.  Steeves, “Zero- 
Base  Planning  and  Budgeting  for  Utilities,”  Public  Utilities 
Fortnightly y 97(9):  53  (1976). 


ZERO-BASE  BUDGETING  AND 
PERFORMANCE  EVALUATION 

Zero-base  budgeting  assumes  an  ability  to 
express  clearly  the  goals  and  objectives  of 
each  decision  unit  being  budgeted,  as  well  as 
the  goals  and  objectives  of  the  entire  organiza- 
tion. Regular  zero-base  review  of  Federal  pro- 
grams would  require  effective  performance 
evaluation  of  program  success  or  failure,  a 
task  of  some  magnitude.  The  Federal  govern- 
ment is  now  spending  at  least  $200  million  a 
year  for  program  evaluation  according  to  the 
US  General  Accounting  Office  (GAO).^  The 
US^  General  Accounting  Office  and  the  Office 
of  Management  and  Budget  (0MB)  have  de- 
veloped program  evaluation  guidelines  to 
assist  Federal  departments  and  agencies.*^ 

Guidelines  have  emphasized  development 
of  appropriate  evaluation  criteria  as  a com- 
ponent of  agency  legislative  proposals  and  as 
an  integral  part  of  the  implementation  plan 
for  new  programs.  In  1975  0MB  developed, 
in  draft  form,  several  program  evaluation 
guidehnes  selectively  illustrated  below 

It  is  apparent  the  Federal  Government 
faces  a difficult  task  in  determining  objectives 
and  carrying  out  evaluations  required  to  im- 
plement ZBB.  Mr.  Charles  H.  Wilson,  a US 
Congressman  from  California,  recently  sum- 
marized some  general  concerns  on  ZBB  and 
program  evaluation: 

“In  summary,  it  should  be  clear  now  that  ZBB, 
cannot  simply  be  imposed  on  the  entire  Federal 
Government.  Programs,  program  goals,  objective, 
and  evaluation  techniques  vary  too  widely.  A 
system  and  methodology  so  flexible  that  it  could 
be  adapted  to  all  programs  would  be  meaning- 
less. . . . any  system  rigourous  enough  to  be  suc- 
cessful in  some  cases  would  certainly  not  be  ap- 
propriate in  others. 

The  management  tool  of  zero-base  budgeting  can 
provide  information  on  alternatives.  But  no  budg- 
eting system  can  make  the  ultimate  decisions  on 
priorities  and  the  allocation  of  resources:  Evalua- 
tion is  still  more  art  than  science  and  is  not  a 
panacea.  The  rational  process  by  itself  cannot 
bear  responsibility  for  criterion,  continuation,  or 
discontinuation  of  governmental  programs.  The 
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GUIDELINES 


CONSIDERATIONS 


1.  Is  the  evaluation  system  an  integral  part 
and  contributory  part  of  the  agency's 
decisionmaking  processes? 


2.  Is  there  evidence  of  adequate  attention  to 
the  administration  of  evaluation  activities? 


• Formal  connection  between  evaluation  system  and 
budget  formulation  and  justification  process. 

• Direct  relationship  between  evaluation  system  and  other 
nianagement  processes  linking  evaluation  results  with 
timetable  for  accomplishing  objectives. 

• Early  consideration  of  evaluation  requirements  during 
program  design. 

• Does  system  maximize  use  of  existing  factual  data? 

• Are  agency  evaluation  units  technically  proficient  to 
to  perform  evaluation? 

• Are  individual  evaluation  projects  planned  and  justified 
in  the  context  of  related  and  alternate  projects? 

• Do  evaluation-related  data  collection  plans  avoid  un- 
necessary paperwork  and  protect  privacy  of  individuals/ 
organizations? 

• Are  evaluation  findings  disseminated? 


3.  Is  there  a central  focal  point  for  evalua- 
tions within  the  department  or  agency? 


• Is  a senior  policy  official  responsible  for  evaluation 
policy  and  criteria? 

• Is  the  focal  point  the  most  appropriate  for  responding  to 
external  requests  for  evaluation? 


Figure  3,  Program  Evaluation  Guidelines 


real  challenge  is  to  policy-makers  to  use  evalua- 
tion effectively,  while  recognizing  uncertainties 
in  information  and  limits  of  analysis.”^ 

SUNSET  LEGISLATION  AND 
ZERO-BASE  BUDGETING 

The  93rd  US  Congress,  in  1974,  passed  the 
Congressional  Budget  and  Impoundment  Con- 
trol Act  of  1974  (Public  Law  93-344)  which 
established  a new  congressional  budget  proc- 
ess, including  House  and  Senate  Budget  Com- 
mittees, a Congressional  Budget  Office  (CBO), 
and  a revised  Federal  Government  Fiscal 
Year,  beginning  October  1,  and  effective 
October  1,  1976.  The  1974  budget  reform  act 
is  causing  Congress,  for  the  first  time,  to  ex- 
amine spending  priorities  with  predetermined 
levels  of  revenue.  Title  VII  of  the  Act  author- 
izes Congressional  Committees  to  conduct 
program  testing  or  analysis  of  agency  activ- 
ities. Title  VII  also  greatly  expanded  the  Gen- 
eral Accounting  Office’s  program  evaluation 
authority. 

In  1976,  Senator  Edmund  S.  Muskie,  (D.), 
Maine,  Chairman  of  the  Senate  Budget  Com- 


mittee, and  Senator  William  V.  Roth,  Jr.,  (R.), 
Delaware,  introduced  in  the  94th  Congress’ 
Senate  Bill  S.  2925,  the  “Government  Econ- 
omy and  Spending  Reform  Act  of  1976,” 
that  proposed  a procedure  for  zero-base  re- 
view and  evaluation  of  Government  programs 
and  activities  every  5 years.  Senate  Bill  S. 
2925  became  known  as  the  “Sunset  Bill”  be- 
cause the  sun  could  set  on  programs  with 
unfavorable  reviews,  and  all  programs  would 
have  finite  expiration  dates  in  the  enabling 
legislation. 

Congressional  leaders  consider  Sunset  re- 
view as  logical  extensions  of  the  new  congres- 
sional budget  process.  Where  the  current 
budget  process  set  spending  priorities.  Sun- 
set reviews  provide  a method  of  determining 
whether  Federal  Programs  are  meeting  these 
priorities. 

The  Muskie-Roth  Bill  would  have  ter- 
minated the  authorization  provisions  for 
nearly  all  Federal  Programs  every  5 years. 
Exempt  from  the  “Sunset  provision”  would 
be  social  security,  other  'pension  programs, 
medicare  and  interest  payments  on  the  na- 
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tional  debt.  Tax  expenditure  provisions 
(generally  defined  in  the  Internal  Revenue 
Code  of  1954)  that  allow  a special  exclusion, 
exemption,  or  deduction  for  determining  tax 
liability  were  also  included  within  the  pro- 
visions of  the  bill,  since  they  are  considered  to 
be  revenue  losses  to  the  US  Treasury. 

Programs  could  not  be  continued  beyond 
their  expiration  dates  unless  Congress  and  the 
Administration  conducted  a program  zero- 
base  review,  1 year  prior  to  scheduled  expira- 
tion date. 

Many  members  of  Congress,  during  Senate 
and  House  hearings  on  the  proposed  Sunset 
legislation,  expressed  concern  over  the  inflexi- 
bility of  the  5-year  reauthorization  period, 
and  the  rigidity  of  the  proposed  procedures.® 
Executive  branch  concerns  focus  on  the  in- 
creased ZBB  paperwork  burden  and  the  ex- 
tended and  involved  decisionmaking  period 
required.  Senate  Bill  S.  2925  and  a com- 
panion House  Bill  (H.R.  11734)  died  with 
the  end  of  the  94th  Congress.  Senator  Muskie 
and  a bipartisan  group  of  41  US  senators,  in- 
troduced on  January  10,  1977,  Senate  Bill 
S.  2,  entitled  “The  Sunset  Act  of  1977.” 
Senate  Bill  S.  2 “requires  authorizations  of 
new  budget  authority  for  Government  pro- 
grams at  least  every  5 years,  to  provide  for 
review  of  Government  programs  every  5 
years,  and  for  other  purposes.”® 

Programs  performing  similar  functions 
would  be  grouped  into  program  decision 
packages,  and  terminate  in  the  same  year,  so 
that  Congress  could  review  them  as  a package. 
The  first  package  to  expire  would  require  re- 
authorization after  September  30,  1979.  Na- 
tional Defense  programs,  functional  category 
050,  would  be  included  in  the  first  review 
period.  Another  set  of  program  decision  pack- 
ages would  follow  at  the  end  of  each  fiscal 
year,  ending  in  Fiscal  Year  1983.  Each  subse- 
quent review  date  applicable  to  a program  is 
the  date  5 years  following  the  preceding  re- 
view date.  The  5-year  process  is  intended  to 
balance  Congressional  Committee  workload. 

Sunset  review  would  determine  if  the 
merits  of  the  programs  justify  continuation 
rather  than  termination,  or  continuation  at 


a level  less  than,  equal  to,  or  greater  than 
the  existing  level.  Senate  BUI  S.  2 differs 
substantially  from  Senate  Bill  S.  2925,  as  a 
result  of  Hearings,  and  an  apparent  philo- 
sophical change  that  emphasizes  Sunset  (in- 
cluding tax  expenditures)  rather  than  ZBB 
provisions. 

To  alleviate  Congressional  concerns,  a US 
House  of  Representatives  Appropriations 
Subcommittee  devised  a limited  test  of  ZBB. 
The  experiment  began  in  late  1976  when  the 
Consumer  Product  Safety  Commission  and 
the  National  Aeronautics  and  Space  Admin- 
istration (NASA)  were  directed  to  prepare 
their  Fiscal  Year  1978  (October  1,  1977 
through  September  30,  1978)  budget  mate- 
rials using  both  the  usual  technique  and 
ZBB  procedures.  The  two  agencies  were 
selected  by  the  Subcommittee  because  tech- 
nical activities  of  these  agencies  could  be 
more  easily  measured  than  some  other  agen- 
cies in  the  Subcommittee’s  jurisdiction. 
“While  it  is  too  early  to  draw  comprehensive 
conclusions,  the  project  has  generated  the 
feeling  among  those  involved  that  it  would 
be  extremely  difficult  to  institute  zero-base 
budgeting  across  the  entire  Government  and 
receive  great  results  the  first  year.”^°  The 
Subcommittee  plans  to  have  the  General 
Accounting  Office  and  the  Congressional 
Budget  Office  evaluate  the  experiment. 

As  shown  in  Figure  4,  the  proposed  Sunset 
Act  of  1977  timetable  may  add  significantly 
to  Congressional  information  demands  from 
the  Department  of  Defense.  The  approach 
builds  on  the  fixed  budget  timetable  adopted 
in  the  Congressional  Budget  and  Impound- 
ment Control  Act  of  1974.^^ 

As  presently  conceived  in  the  Federal  gov- 
ernment, ZBB  is  primarily  a managerial  in- 
strument of  executive  budgeting,  while  Sun- 
set, especially  as  defined  in  Senate  Bill  S.  2,  is 
linked  to  law-making  functions  and  becomes 
a legislative  device.*  Zero-base  budgeting  and 


♦For  a comparison  of  the  features  and  uses  of  ZBB/ 
Sunset  by  the  Executive  and  Legislative  branches  of  gov- 
ernment, see  the  testimony  of  Allen  Schick,  Congressional 
Research  Service,  in  “Zero-Base  Budget  Legislation,’*  Ref- 
erence 8,  p.  51. 
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ON  OR  BEFORE 


ACTION 


September  1977  House  and  Senate  Budget  Committees  report  to  Congress  on 

relationship  between  tax  expenditure  provisions  and  related 
programs. 

January  1,  1978  General  Accounting  Office/Congressional  Budget  Office  with 

OMB  submit  to  Congress  analysis  of  all  programs  including 
enabling  legislation.  Joint  Committee  on  taxation  identifies/ 
recommends  tax  expenditure  provision  termination  dates, 

March  1,  1978  House  and  Senate  Budget  and  Appropriations  Committees 

complete  review  of  report. 

October  1978  General  Accounting  Office  furnishes  Congressional  Com- 

mittees results  of  prior  audits  for  programs  and  tax  expend- 
iture provisions  included  in  first  annual  Sunset  review.  Fiscal 
year  begins. 


May  15,  1979 Congressional  Committees  complete  and  submit  reports  on 

each  program  to  the  House  or  Senate. 

September  30,  1979 Congress  completes  first  annual  program  Sunset  review. 

September  30,  1980 Congress  completes  2nd  annual  program  Sunset  review.  Cit- 

izens' Commission  on  the  Organization  and  Operation  of 
Government  final  report  to  President/Congress. 

September  30,  1981 Congress  completes  third  annual  program  Sunset  review. 

September  30,  1982 Congress  completes  fourth  annual  program  Sunset  reviews. 

September  30,  1983 Congress  completes  fifth  annual  program  Sunset  review. 

(Repeat  cycle  as  required.) 


Figure  4,  Proposed  Sunset  Act  Timetable 

Source:  Senate  Bill  S.  2,  95th  Congress 
1st  Session 


Sunset  Legislation  use  different  but  comple- 
mentary procedures,  and  share  a common 
objective:  to  compel  periodic  review  of  every 
Federal  program  to  determine  whether  it 
should  be  continued. 

ZERO-BASE  BUDGETING  AND 
THE  DOD  PLANNING, 
PROGRAMMING,  BUDGETING 
SYSTEM 

The  Office  of  Management  and  Budget 
issued  to  all  agencies  in  early  1977  ZBB  in- 
structions* applicable  to  the  preparation, 
analysis,  and  justification  of  fiscal  year  1979 
budget  requests. The  required/optional 
decision  package  set  is  illustrated  in  Figure  5 ; 


*Directed  to  the  Heads  of  Executive  Departments  and 
Establishments. 


Proposed  changes  (supplemental,  amend- 
ments, recissions)  in  current  year  amounts, 
and  new  programs  or  activities  are  to  be  pro- 
posed in  separate  decision  packages. 

Department  of  Defense  compliance  with 
the  Office  of  Management  and  Budget  ZBB 
requirements  should  be  facilitated  by  the 
“Five  Year  Defense  Program  (FYDP)”  which 
aggregates  information  at  the  program  (Stra- 
tegic Forces,  e.g.)  and  package  (Minuteman 
Squadrons,  e.g.)  levels;  and  information  re- 
quired by  the  Planning,  Programming,  Budget- 
ing System  structure  and  related  DOD  policy 
documents.^2'^^'^®  As  ZBB  implementation 
proceeds,  DOD  resource  management  sys- 
tems may  require  modification  to  respond 
to  a revised  budgetary  process. 

As  recently  pointed  out  by  the  Assistant 
Secretary  of  Defense-Comptroller,  ZBB  dif- 
fers in  three  major  respects  from  past  DOD 
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Current  Represents  continuance 

Level  of  previous  year's  level 

Required 

Minimum  The  level  below  which 
Level  it  is  not  feasible  to 

continue. 

A level  or  levels  between  the  min- 
imum and  current  levels 

Optional 

Any  additional  increments  desired 
above  the  current  level. 

Figure  5,  Decision  Package  Set 


PPBS  practices:  “First,  there  will  be  many 
more  decision  packages  than  in  the  past. 
Military  components  will  submit  several  (not 
one)  decision  packages  for  each  (decision) 
unit,  and  the  options  to  be  considered  also 
will  increase  in  number.  Second,  the  package 
for  the  lowest  resource  level  must  include  an 
explicit  zero-base  justification.  And,  third, 
the  package  must  be  ranked,”^® 

PRESIDENT  CARTER’S  VIEWS 

In  the  January  1977  issue  oiNation*s  Busi- 
ness''^ President  Carter  described  why  he 
will  use  zero-base  budgeting  during  his  ad- 
ministration. 

“From  my  experience  in  government,  as  well  as 
the  experience  of  corporations  in  the  business 
world,  a number  of  clear-cut  benefits  from  an 
effective  zero -base  budgeting  effort  can  be  cited. 
These  benefits  include: 

• Focusing  the  management  process  on  anal- 
ysis and  decisionmaking,  rather  than  simply 
on  numbers; 


• Combining  planning,  budgeting,  and  opera- 
tional decisionmaking  into  one  process; 

• Forcing  managers  to  evaluate  in  detail  the 
cost-effectiveness  of  their  operations; 

• Providing  a system  to  trade-off  between  long- 
term and  short-term  needs  during  the  budget- 
ing period,  as  well  as  a follow-up  tool  on  cost 
and  performance  during  the  year; 

• Allowing  for  quick  budget  adjustments  or  re- 
source shifts  during  the  year,  if  necessary, 
when  revenue  falls  short ; 

• Identifying  similar  functions  among  different 
departments  for  comparison  and  evaluation; 

• And,  most  important  to  me,  broadly  expanding 
management  participation  and  training  in  the 
planning,  budgeting,  and  decisionmaking 
process.” 

IMPLICATIONS  FOR 
THE  FUTURE 

The  President  has  stated  his  commitment 
to  ZBB,  reorganizing  government  to  improve 
efficiency,  and  providing  increased  citizen 
access  to  government.  Since  Watergate,  gov- 
ernment has  lost  some  credibility,  and  there 
is  growing  concern  about  continued  growth 
of  pub  he  sector  spending  programs.  The  new 
budgetary  process  adopted  by  the  Congress  in 
1974  is  one  reflection  of  these  concerns,  and 
there  is  considerable  support  for  budget  re- 
form, reflected  by  ZBB  and  Sunset  Legisla- 
tion. Although  the  immediate  imposition  of 
ZBB  and  Sunset  reviews  throughout  the 
Federal  Government  may  be  unlikely,  current 
Legislative  and  Executive  Branch  budget  re- 
form efforts  presage  a coming  major  change  in 
the  US  Government’s  budget  process.  □ 
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The  Army  Budget 

and 

Combat  Capability 

by 

Mr.  Leonard  S.  Freeman,  DAC,  US  Army  Concepts  Analysis  Agency 


The  US  Army  Concepts  Analysis  Agency  is  developing  a quantitative  means  by  which  to 
assess  changes  in  the  Army’s  combat  capabilities  resulting  from  real  or  expected  changes 
m Army  budgets.  This  effort  incorporates  both  an  econometric  model  of  the  Army  and  a 
theater  combat  simulation  model  within  an  overall  methodology  framework.  The  linking 
of  the  fiscal  model  with  a combat  effectiveness  model  is  a unique  feature  of  this  project. 


INTRODUCTION 

The  Army’s  budget  and  the  Army’s  com- 
bat capability  must,  logically,  be  related 
to  one  another.  An  Army  program  is  devel- 
oped to  counter  a threat,  improve  readiness 
and  attain  a myriad  of  other  objectives.  Fiscal 
arrangements  are  then  designed  to  support 
the  program.  It  follows  that  if  the  budget  is 
changed  the  capability  will  change.  This 
interaction  was  a concern  of  the  then  Under 
Secretary  of  the  Army,  Norman  Augustine, 
who,  in  November  1975,  requested  that  the 
US  Army  Concepts  Analysis  Agency  (CAA) 
define  an  approach  to  link  the  budget  to  a 
measure  of  combat  capability.  The  project 
was  accomphshed  in  slightly  over  3 months 
time  and  is  referred  to  as  the  100-Dav 
Study. 

The  100-Day  Study  provided  a methodology 
for  ascribing  effectiveness  to  a budget.  It 
showed  that  there  was  a way  to  measure 
combat  capability  changes  resulting  from 
budget  changes.  It  also  indicated  that  effec- 
tiveness per  budgeted  dollar  factors  could  be 
determined  which  would  be  useful  in  con- 
sidering budget  alternatives.  The  method- 


ology proposed  in  the  1 00-Day  Study  was 
favorably  received  but  the  concept  required 
refinement  and  testing  before  it  could  be 
made  available  to  the  Army  Staff.  The  Army 
Dollar  Resource  Allocation-Phase  II  (ADRA 
II)  Study  evolved  from  that  effort.  The 
ADRA  II  Study  results  in  an  analytical 
mechanism  for  use  by  the  Army  Staff  in 
formulating  and  supporting  Army  plans, 
programs  and  budgets.  This  article  presents 
the  basic  ADRA  methodology  and  proposes 
potential  applications. 


MAJOR  ASSUMPTIONS 

The  following  major  assumptions  were  ap- 
plied to  make  the  problem  tractable: 

• The  Army  can  be  represented  as  an 
economic  system  having  producing  and 
consuming  sectors. 

• Input-output  analysis,  an  econometric 
modehng  technique,  is  appropriate  for 
use  in  describing  the  interrelationships 
within  the  Army. 


Vol.  I,  No.  4 


45 


METHODOLOGY 

The  Army  is  viewed  as  an  economic  system 
consisting  of  producing  and  consuming  sec- 
tors. This  Army  economy  is  modeled  using 
the  techniques  of  input-output  theory.^ 

The  input-out  model  thus  derived  is  utilized 
to  compute  how  the  dollars  in  the  Army  out- 
out  sectors  (e.g.,  the  division  forces)  change 
when  the  input  (budget)  changes.  Also  the 
theory  can  be  applied  to  compute  the  budget 
required  to  assure  a desired  level  of  dollars  in 
the  output  sectors. 

In  a separate  analysis  a value  for  an  effec- 
tiveness measure  was  estabUshed.  The  measure 
used  is  the  Weighted  Unit  Value  (WUV)  score 
which  is  a value  based  on  the  number  of 
weapons  in  a force,  and  the  firepower,  ma- 
neuverabiUty,  survivability  and  other  charac- 
teristics of  these  weapons.  The  WUV  scores, 
in  Army  Dollar  Resource  Allocation,  are  com- 
puted for  the  force  projected  in  the  Army 
Program  Objective  Memorandum  (POM)  and  a 
separate  value  is  computed  for  each  year  of 
the  POM  period.  This  effectiveness  measure  is 
used  in  the  computation  of  effectiveness  per 
dollar  factors  and  as  a primary  input  to  a dy- 
namic, macro-level  combat  simulation  model; 
the  model  provides  an  assessment  of  the 
Army’s  combat  capability  in  a North  Atlantic 
Treaty  Organization,  Central  Europe,  mid- 
intensity war. 

The  methodology  is  synopsized  in  Figure 
1.  Starting  with  data  contained  in  the  Five 
Year  Defense  Program  (FYDP)  and  the  POM, 
a base  case  fiscal  arrangement  and  force  ef- 
fectiveness measure  (WUV  score)  are  com- 
bined to  produce  effectiveness  per  dollar 
factors  for  each  Army  sector  and  appropria- 
tion. All  dollar  figures  in  the  Five  Year  De- 
fense Program  are  converted  to  constant 
dollars  to  negate  the  influence  of  inflation. 
Also  a combat  simulation  is  performed  to  esti- 
mate the  war  fighting  performance  of  the 
base  case  force.  The  process  just  described 
constitutes  calibrating  the  Army  Dollar  Re- 
source Allocation  model  to  the  base  case. 
The  results  of  this  calibration  are  required 
as  a foundation  from  which  to  assess  the  im- 


*See  references  for  discussions  of  input-output  theory. 


pact  of  changes  in  Army  capability  resulting 
from  changes  in  the  budget.  To  examine  a 
budget  change,  factors  are  applied  to  com- 
pute the  dollar  changes  at  the  output  sector. 
Those  output  changes,  together  with  the 
previously  computed  effectiveness  per  dollar 
factors  allow  the  overall  change  in  effective- 
ness (WUV)  to  be  computed.  To  complete  the 
procedure,  the  revised  WUV  score  resulting 
from  the  dollar  change  is  input  to  the  combat 
simulation  model  and  the  resulting  combat 
capability  is  compared  to  that  of  the  base 
case. 

THE  ADRA  PROCESS 

The  description  contained  in  the  preceding 
paragraph  was  a general  review  of  the  ADRA 
methodology.  The  Army  Dollar  Resource 
Allocation  information  process  is  shown  in 
Figure  2.  The  Five-Year  Defense  Program  con- 
tains the  dollars  associated  with  each  Army 
program  element.  A set  of  sector  composition 
rules  is  applied  to  compute  the  dollars  con- 
tained in  each  Army  economic  sector.  That 
money  is  distributed  throughout  the  Army 
economy  by  use  of  a set  of  fiscal  allocation 
rules.  Then  a set  of  WUV  allocation  rules  is 
applied  to  associate  the  effectivenss  measure 
with  a doUar  value  for  each  sector  and  ap- 
propriation. The  material  which  follows  pro- 
vides greater  detail  regarding  the  Army 
sectors  and  their  composition,  the  distribu- 
tion of  dollars,  and  the  computation  of  the 
effectiveness  per  dollar  values. 

Army  Sectors 

The  sectors  of  the  Army  economy  are  ex- 
tensions of  the  ten  Five-Year  Defense  Pro- 
grams; see  Table  1.  For  instance.  Program  3 
(Intelligence  and  Communication)  was  con- 
sidered as  constituting  two  sectors:  the  In- 
telligence sector,  and  the  Communications 
sector.  It  was  reasoned  that  for  the  concept 
of  economic  sectors  to  gain  acceptance  and 
provide  maximum  utility,  the  sectors  should 
be  terms  familiar  to  Army  financial  planners 
and  managers— the  Five-Year  Defense  Pro- 
grams met  that  criterion.  The  only  exception 
was  the  Base  Operations  sector.  Base  Opera- 
tions is  not  a Five  Year  Defense  Program  but 
it  is  a highly  visible  and  integral  facet  of  the 
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Figure  1.  Synopsis  of  Army  Dollar  Resource  Allocation  Methodology. 
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Table  1.  Sectors  of  the  Army  Economy 
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Figure  2.  Army  Dollar  Resource  Allocation  Informa- 
tion Process. 
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Army,  receiving  considerable  attention  and 
scrutiny  from  the  Office  of  the  Secretary  of 
Defense  and  the  Congress;  therefore  it  was 
established  as  a separate  sector. 

Sector  Composition 

After  having  estabhshed  the  sectors  of  the 
Army  economy,  it  was  necessary  to  determine 
the  individual  sector  composition  or  construc- 
tion in  order  to  associate  a dollar  value  with 
each  sector.  Again,  the  Five  Year  Defense  Pro- 
gram was  used  to  aid  this  process.  There  are 
approximately  700  program  elements  in  the 
Army’s  portion  of  the  Five  Year  Defense 
Program  and  each  program  element  is  defined 
in  DOD  Handbook  7045. 7-H.^  A Department 
of  the  Army  working  group  reviewed  each 
program  element  and  assigned  it  to  one  of  the 
19  Army  sectors.  Since  the  Five  Year  Defense 
Program  contains  fiscal  data  by  appropriation 
for  each  program  element,  summing  the  dol- 
lars associated  with  the  program  elements  in  a 
sector  provides  the  budget  for  each  sector,  by 
appropriation. 

Dollar  Distribution 

The  next  step  in  the  methodology  requires 
distributing  the  budget  for  each  sector 
throughout  the  Army.  To  distribute  the 


48 


Defense  Systems  Management  Review 


budget,  each  sector  is  considered  a producer — 
its  budget  allows  the  sector  to  produce  goods 
or  services  available  to  the  Army  economy^ 
each  sector  is  also  considered  a consumer— it 
creates  demands  for  the  goods  or  services  pro- 
duced by  the  sectors.  A set  of  dollar  alloca- 
tion rules  is  used  to  distribute  the  dollars 
from  the  producers  to  the  consumers. 

The  process  of  distributing  the  dollars 
throughout  the  Army  economy  is  illustrated 
in  Figure  3.  Consider  a simplified  Army  con- 
sisting of  4 sectors  one  of  which  is  the  Train- 
ing sector.  Further  assume  that  the  Training 
sector  is  comprised  of  two  program  elements: 
recruit  training  and  specialized  training.  For 
any  appropriation,  the  sum  of  the  budgets 
for  the  two  program  elements  constitutes  the 
budget  for  the  Training  sector.  That  sector 
budget  is  spent  to  train  personnel  in  each  sec- 
tor (including  the  training  sector  since  train- 
ing personnel  must  also  be  trained).  If  it  is 
assumed  that  the  Training  sector  allocates 
its  resources  according  to  the  number  of 
people  in  each  sector,  then  the  training 
budget  can  be  distributed  from  the  Training 
sector  to  all  sectors  in  proportion  to  the  per- 
sonnel in  each  sector.  Each  producing  sector’s 
budget  can  be  distributed  by  a rationale 
that  reflects  the  goods  or  services  provided 
by  the  producer’s  dollars. 

The  dollar  distribution  procedure  is  re- 
peated for  each  appropriation  using  separate 
allocation  rules  to  reflect  the  different  goods 
or  services  provided  by  each  appropriation 
(e.g.,  the  Aircraft  Procurement  appropriation 
can  be  used  only  to  buy  aircraft  related  parts 
and  equipment-it  may  not  be  used  to  pay 
salaries;  Military  Personnel,  Army  (MPA) 
money  is  used  only  for  active  duty  military— 
it  cannot  be  used  to  buy  hardware).  The  re- 
sult of  distributing  the  entire  budget  (all 
sectors,  all  appropriations)  is  portrayed  in 
Figure  4. 

Dollars  for  Output.  If  the  rightmost  col- 
umn of  Figure  4 represents  the  output  sector 
of  the  Army,  then  the  shaded  area  contains 
that  portion  of  the  budget  demanded  (or 
consumed)  by  the  output  sector.  That  is,  it 
contains  the  dollars  from  each  sector  (by  ap- 
propriation) allocated  in  support  of  the  out- 
put sector. 
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Note:  Total  sector  dollars  are  distributed  across  Army  according  to  allocation  rules. 

Figure  3.  Dollar  Distribution 


Effectiveness  per  Dollar.  Applying  the 
methodology  just  described  yields  the  dol- 
lars, by  sector  and  appropriation,  that  support 
the  division  forces.  As  indicated  previously, 
the  effectiveness  measure  of  the  force  is  ex- 
pressed in  terms  of  WUV.  The  WUV  score  is 
linked  to  the  sectors  and  appropriations 


Figure  4.  Total  Budget  Distribution 
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utilizing  a methodology  that  relates  WUV 
to  the  dollar  distribution.  First,  the  WUV  as- 
sociated with  each  sector  is  computed.  Then 
each  sector’s  WUV  value  is  distributed  among 
the  appropriations  that  comprise  the  sector. 
Dividing  the  resultant  WUV  by  the  appropria- 
tion dollars  yields  a WUV  per  dollar  value  for 
each  sector-appropriation  pair.  Arraying  these 
WUV  per  dollar  values  from  highest  to  lowest 
produces  an  ordered  list  that  can  be  used  to 
assess  the  impact  of  budget  change  since  the 
product  of  the  dollar  change  to  a sector-ap- 
propriation times  its  WUV  per  dollar  factor 
provides  the  change  in  WUV.  The  list  can  also 
reveal  areas  of  high  or  low  dollar  leverage  in 
terms  of  combat  capability.  Sector-appropria- 
tion pairs  with  high  WUV  per  dollar  factors 
would  be  prime  candidates  for  increased  fund- 
ing while  reductions  would  be  preferable  in 
those  sector-appropriation  pairs  with  low 
factors.  The  computation  of  the  ordered  list 
is  the  final  step  in  establishing  a base  case. 

POTENTIAL  USES 

The  ADRA  methodology  may  be  applied 
throughout  the  program/budget  cycle.  Uses 
which  appear  to  offer  greatest  potential  bene- 
fit are  given  here. 

Identification  of  Leverage  Points 

A budget  and  an  associated  force  effec- 
tiveness (WUV)  score  are  selected  to  establish 
a base  case.  An  example  could  be  the  budget 
request  which  the  Department  of  the  Army 
submits  to  the  Office  of  the  Secretary  of  De- 
fense each  October.  As  previously  indicated 
the  budget  and  the  WUV  score  computed  for 
the  Army  are  combined  to  produce  a WUV 
per  dollar  list  which  is  arranged  in  descend- 
ing numerical  order.  The  list  can  be  used 
directly  to  identify  areas  of  high  or  low  lev- 
erage in  terms  of  combat  capability  (i.e., 
determine  prime  candidates  for  dollar  incre- 
ments or  decrements  to  have  the  most  favor- 
able impact  on  combat  effectiveness).  The 
list  is  necessary  to  assess  the  impact  of  dol- 
lar changes. 

Assessment  of  Directed  Change 

Changes  to  the  base  case  budget  may  be 


directed  by  the  Army  Staff  or  higher  au- 
thority. Review  of  the  change  rationale  can 
reveal  the  sectors  and  appropriations  affected. 
For  example,  if  the  Operations  and  Mainte- 
nance, Army  (OMA)  budget  for  the  Supply 
sector  is  to  be  reduced  by  a given  amount, 
then  that  dollar  reduction  multiplied  by  the 
WUV  per  dollar  factor  for  OMA-Supply 
yields  the  resultant  change  in  WUV . A repeat 
of  this  operation  for  each  affected  appropriation- 
sector  pair  provides  the  total  change  in  WUV 
associated  with  specific  changes  in  the  budget. 
Then  operating  the  combat  simulation  model 
with  the  revised  WUV  score  allows  the  impact 
of  the  dollar  change  to  be  assessed  in  terms  of 
combat  capability. 

Formulation  of  Alternatives 

The  ability  to  prepare  alternatives  to  dollar 
changes  directed  by  higher  authority  is  inher- 
ent to  the  ADRA  methodology.  Such  alterna- 
tives can  be  constructed  to  retain  the  highest 
possible  combat  capability  in  the  face  of 
pressures  for  dollar  reduction.  The  amount  of 
real  or  expected  changes  to  the  base  case 
budget  is  the  input  required  to  prepare  alter- 
natives. The  ADRA  methodology  generates 
alternatives  based  on  the  ordered  list  of  WUV 
per  dollar  factors.  If,  for  example,  an  alterna- 
tive way  to  absorb  a $1 00  million  cut  is  desired, 
the  lowest  WUV  per  dollar  appropriation- 
sector  combinations  would  seem  appropriate 
for  consideration.  By  absorbing  a cut  starting 
with  the  elements  at  the  bottom  of  the  WUV 
per  dollar  list,  an  alternative  can  be  generated 
that  minimizes  the  detriment  to  Army  com- 
bat capability  (e.g.,  an  alternative  with  the 
smallest  reduction  in  WUV).  If  an  increase  in 
budget  is  contemplated,  money  could  be 
added  by  appropriation-sector  starting  at  the 
top  of  the  WUV  per  dollar  list  to  most  favor- 
ably impact  combat  capability.  Such  alterna- 
tives would  place  additional  money  into  the 
elements  that  would  produce  the  highest 
WUV. 

Application  of  Fences 

When  considering  alternatives,  whether 
increasing  or  decreasing  the  budget,  fences 
(i.e.,  fixed  dollar  levels)  can  be  applied  at  the 
sector,  appropriation  or  combined  sector- 
appropriation  level.  Thus,  if  an  alternative 
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to  a budget  cut  is  being  generated,  the  Army 
Dollar  Resource  Allocation  system  can  be 
instructed  to  refrain,  for  example,  from  de- 
leting any  Military  Personnel,  Army  (MPA) 
dollars,  or  to  delete  no  more  than  $10  mil- 
lion from  the  OMB-Base  Operations  budget; 
the  number  of  fences  is  unlimited  within  the 
Army  Dollar  Resource  Allocation  framework. 
The  utility  of  this  feature  is  that  it  allows  the 
construction  of  alternatives  that  reflect 
realistic  and  desirable  constraints  within  the 
Army  program/budget  structure. 

SUMMARY 

Based  upon  the  analysis  performed  thus 
far  in  the  Army  Dollar  Resource  Allocation 
project,  the  following  points  have  been  ob- 
served: 

• The  employment  of  the  ADRA  meth- 
odology is  feasible  and  potentially  use- 
ful in  the  Army  program  and  budget 
cycle.  The  impact  of  real  or  expected 
fiscal  changes  can  be  evaluated  in  terms 
of  combat  capabihty  in  a single  year  or 
over  a series  of  years. 

• Alternative  programs  or  budgets  can  be 
generated  and  evaluated.  The  choices  can 
reflect  fences  (fixed  levels)  or  limits 
(floors,  ceilings)  on  budget  adjustments 
to  preserve  Army  capabilities  or  options. 

• Quantitative  arguments  can  be  developed 
to  help  defend  the  Army  program  and 
budgets  from  detrimental  cuts;  multiple 
choices  can  be  prepared  rapidly  with  the 
Army  Dollar  Resource  Allocation  opera- 
tional system. 

• Tests  with  data  from  past  Program/ 
Budget  Decision  cycles  indicate  that 
faced  with  dollar  reductions,  the  more 
control  the  Army  retains  over  how  to 
absorb  the  cut,  the  less  detrimental  is  the 
impact  on  combat  capability.  If  the 
Army  is  permitted  to  select  both  the 
sectors  and  the  appropriations  being  re- 
duced, a higher  effectiveness  can  result 
than  when  only  the  sectors  can  be  desig- 
nated by  the  Army. 


The  ADRA  methodology  is  embodied  in  an 
automated  data  processing  system  that  per- 
mits rapid  turnaround  to  facilitate  formula- 
tion and  evaluation  of  alternative  Army  pro- 
pams  and  budgets.  The  ADRA  II  study  effort 
incorporates  both  an  econometric  model  of 
the  Army  and  a theater  combat  simulation 
model  within  an  overall  methodology  frame- 
work. This  linking  of  the  fiscal  model  with  a 
combat  effectiveness  model  is  a unique  fea- 
ture of  the  Army  Dollar  Resource  Allocation 
effort  allowing  the  impact  of  budget  changes 
to  be  assessed  quantitatively  in  terms  of  com- 
bat capability.  Each  of  these  constituent 
models  address  key  parameters  at  aggregate 
or  macro  levels  of  activity:  the  input-output 
structure  employed  for  the  econometric 
model  represents  an  aggregation  of  budget 
components  of  the  Army;  the  combat  simu- 
lation model  aggregates  military  activity  for  a 
peater-wide  campaign.  Consequently,  the 
interface  of  these  models  provides  for  macro 
analytic  treatment  of  the  linkage  between 
Army  budgets  and  combat  effectiveness.  Ap- 
phcations  of  the  ADRA  methodology  can  be 
targeted  to  levels  of  analysis  consistent  with 
those  of  the  major  component  models. 

WHAT  NEXT? 

The  methodology  described  in  this  article 
is  being  programmed  for  operations  on  auto- 
mated data  processing  equipment  of  the  US 
Army  Management  Systems  Support  Agency 
in  the  Pentagon.  This  will  provide  the  Army 
Staff  a responsive  in-house  abihty  to  use  the 
Army  Dollar  Resource  Allocation  system. 
Over  the  next  year,  extensive  operational 
testing  will  be  accomplished  to  guage  the 
adequacy  of  the  system  design  and  establish 
its  usefulness  in  the  Planning,  Programming 
and  Budgeting  System.  Application  will  be 
tested: 

• In  the  Major  Issue  Cycle  (Jun-Aug) 

• In  the  Program/Budget  Decision  Cycle 
(Nov-Dec) 

• In  Program  Formulation  (Jan-Feb) 

By  April  1978  the  Army  will  have  completed 
developing  and  testing  the  ADRA  method- 
ology. 
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Establishing  the  FAE II 


by 

Mr.  James  A.  Bowen,  Department  of  Navy 
and 

Captain  Ronald  S.  Fry,  Department  of  Air  Force 


As  greater  emphasis  is  given  joint  and  triservice  national  defense  programs,  service  coopera- 
tion becomes  increasingly  important.  In  this  article  the  FAE  II  Program,  based  on  fuel-air 
explosive  technology,  is  reviewed.  The  difficulties  stated,  both  technical  and  procedural,  are 
typical  of  those  encountered  in  the  beginning  phases  of  any  functioning  joint  service  de- 
velopment program.  The  resolutions  agreed  upon  by  the  Navy-Air  Force  team  to  facilitate 
working  relations  are  presented  and  insights  are  provided  for  those  who  become  engaged  in 
future  joint  service  development  programs. 


FUEL-AIR  EXPLOSIVE 
TECHNOLOGY 

A fuel-air  explosive  (FAE)  weapon  is  a 
blast-producing  device  that  has  minimum 
fragmentation  and  incendiary  effects.  The 
results  of  a fuel-air  explosive  detonation  can 
be  characterized  by  a homogeneous  effects 
region  around  the  functioning  point  and  a 
short,  predictable  cutoff  range  from  the 
region  of  effectiveness  to  a region  of  complete 
safety.  The  detonation  of  a fragmentation 
munition,  by  contrast,  gradually  decays  in 
effectiveness  from  the  region  near  the  war- 
head. Hence  there  is  a probability  of  damage, 
although  low,  at  great  distances  from  the  im- 
pact point.  The  fuel-air  explosive  attribute  of 
limited  cutoff  range  is  viewed  favorably  by 
friendly  ground  forces  as  ordnance,  on  oc- 
casion, must  be  employed  close  to  friendly 
positions.  Too,  the  limited  cutoff  range  is 


important  when  targets  must  be  engaged  that 
are  close  to  nontargets  (such  as  churches  or 
schools)  or  when  targets  are  near  to  enemy 
equipment  and  facilities  that  could  be  used 
by  capturing  forces. 

A fuel-air  warhead  differs  from  a conven- 
tional explosive-filled  war  head  in  that  the 
oxidizer  necessary  for  detonation  can  be 
obtained  from  the  air.  An  FAE  device  carries 
only  its  fuel.  Thus  more  energy  can  be  ob- 
tained within  the  given  volume  or  weight 
envelop  of  the  weapon. 

Both  the  Navy  and  the  Air  Force  pursued 
research  and  technology  efforts  leading  to 
the  development  of  first-generation  weapons. 
The  CBU-55  used  by  the  Air  Force  and  the 
Navy,  and  by  the  Republic  of  Vietnam,  was 
developed  and  produced  by  the  Navy.  The  Air 
Force  developed  the  BLU-76,  that  underwent 
examination  in  Vietnam  (the  BLU-76  was  not 
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a production  model).  Both  the  CBU-55  and 
the  BLU-76  required  large  parachutes  to  slow 
the  weapons  to  the  low  velocity  then  neces- 
sary to  obtain  reliable  function.  Weapon  de- 
signs of  this  first  generation  necessitated  re- 
striction of  aircraft  delivery  envelop  and  made 
accurate  delivery  nearly  impossible.  Long 
times  of  fall,  resulting  from  the  slow  velocity, 
allowed  wind  drift  to  deflect  the  weapon 
from  its  target. 

THE  FAE  II  PROGRAM 

The  FAE  II  was  envisioned  as  a second  gen- 
eration fuel  air  explosive  weapon  designed  to 
overcome  dehvery  and  accuracy  limitations. 

Both  the  Navy  and  Air  Force  conducted 
exploratory  and  advanced  development  ef- 
forts seeking  solutions  to  the  problem  of 
high-velocity  functioning  of  advanced  FAE 
weapons.  A number  of  formidable  problems 
resulted  from  the  requirement  for  weapon 
function  at  high  and  variable  terminal  ve- 
locities. Each  service  approached  the  prob- 
lems differently  and  strong  opinions  were 
formed  over  the  best  method  of  achieving  a 
functional  weapon.  Opinion  also  differed 
over  the  state  of  the  art  in  FAE  technology. 
The  Air  Force  adopted  a conservative  ap- 
proach and  the  Navy  advocated  prompt 
entry  into  engineering  development.  The 
process  of  resolving  these  differences  played 
a dominant  role  in  the  FAE  II  Program  and 
had  a major  impact  on  the  working  relations 
established  between  the  services. 

LEAD  SERVICE 
DETERMINATION  AND 
JSOR  GENERATION 

The  Department  of  Defense  has  become 
increasingly  involved  with  reducing  prolifera- 
tion of  weapon  systems  while  reducing  waste 
associated  with  duplication  of  development 
efforts.  Early  in  1971  harmonization  require- 
ments were  generated  that  estabhshed  the 
framework  for  joint  service  programs.*  The 
Air  Munitions  Requirements  and  Develop- 
ment (AMRAD)  Committee  was  estabhshed 
under  the  Director  of  Defense  Research  and 
Engineering**  to  ensure  coordination  of  re- 


quirements. By  1973  it  was  apparent  that  the 
FAE  II  development  efforts  of  the  Navy  and 
Air  Force  were  candidates  for  a joint  program. 

(Note;  The  FAE  II  was  not  a major  pro- 
gram. The  existing  procedural  framework*** 
for  joint  service  projects  had  been  written  for 
major  programs  and  called  for  charters,  pro- 
gram offices  at  high  organizational  levels, 
memoranda  of  understanding,  and  many 
other  items  not  consistent  with  organizations 
and  budgets  available  for  less-than-major 
projects.  Little  useful  procedural  guidance 
was  available  for  setting  up  a FAE  II  joint 
service  development  program.) 

The  task  then  addressed  was  the  selection 
of  the  lead  service  for  the  FAE  II  program. 
No  firm  prerequisites  or  requirements  exist 
that  must  be  satisfied  by  a service  prior  to 
its  being  selected  the  lead  service.  The  guide- 
lines imply  that  the  service  staffing  the 
Requirements/Objectives  (R/0)  document 
through  the  system  to  the  AMRAD  Com- 
mittee has  the  best  opportunity  to  become 
lead  service.  In  the  case  of  the  FAE  II  willing- 
ness by  Navy  headquarters  to  support  the 
program  counted  heavily  in  the  selection  of 
Navy  as  the  lead  service. 

The  Joint  Service  Agreement  on  Harmon- 
ization of  Service  Requirements  and  Char- 
acteristics for  Air  Munitions  states  that  a 
R/0  should  be  prepared  about  1 year  before 
initiation  of  engineering  development.  The 
technical  prerequisites  for  entering  engineer- 
ing development  are  not  defined  beyond  the 
general  statement  that  “feasibility  must  have 
been  demonstrated.”  The  objective  of  ad- 
vanced development  is  to  resolve  unknowns 
and  verify  that  the  technical  and  economic 
bases  for  initiating  engineering  development 

♦Department  of  Defense  Research  and  Engineering  (DDR&E). 
Joint  Service  Agreement:  “Harmonization  of  Service  Qual- 
itative  Requirements  and  Characteristics  for  Air  Munitions, 
(DDR&E  memo  dtd.  27  Jan  7 1). 

** Joint  Service  Agreement  on  Harmonization  of  Service 
Requirements  and  Characteristics  for  Air  Munitions  signed 
by  Secretary  of  Defense  Packard,  30  Apr  7 1 . 

^♦♦Department  of  the  Navy,  NAVMAT  Instr.  5000.10 A, 
“Air  Force,  Army  and  Navy  Agreement  on  the  Management 
of  Multi-Service  Systems,  Programs,  and  Projects,”  4 Sep  73. 
(superseding NAVMAT  Instr.  5000.10  dtd  25  Apr  68.) 
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of  the  weapon  system  are  valid.  Demonstra- 
tion of  feasibility  during  advanced  develop- 
rnent  may  be  subject  to  various  interpreta- 
tions ranging  from  a convincing  engineering 
analysis  to  a functioning  prototype  device.  In 
general  a brassboard  and  advanced  prototyp- 
ing effort  is  necessary  to  confirm  that  the 
technology  is  feasible  and  that  the  design 
concept  has  military  value  and  utility.  A Judg- 
ment must  be  made  as  to  whether  the  feasi- 
bility demonstration  indicates  acceptable  risk 
in  proceeding  to  the  next  phase  of  develop- 
ment. Whether  a risk  is  acceptable  or  not  is 
influenced  by  many  factors  difficult  to 
quantify. 

With  the  stated  prerequisites  as  a guide,  the 
Navy  submitted  the  FAE  II  R/0  document  to 
the  AMRAD  Committee  in  March  of  1973. 
The  submittal  was  justified  on  the  basis  of 
work  completed,  leading  to  the  belief  that 
engineering  development  could  be  initiated 
within  1 year.  This  action  was  considered  pre- 
mature by  the  Air  Force  development  group 
at  Eglin  Air  Force  Base.  The  Air  Force  posi- 
tion was  based  on  the  opinion  that  neither 
service  would  be  able  to  demonstrate  a func- 
tional prototype  device  prior  to  the  engineer- 
ing development  proposed  start  date. 

On  24  April  1973,  the  AMRAD  Committee 
transmitted  the  Navy  R/0  document  to  the 
other  services  requesting  comments  and  rec- 
ommendations.* The  R/0  document  con- 
tained a preliminary  operational  requirement 
that  called  for  a single  1 ,000-pound  class 
weapon.  On  1 August  1973,  the  AMRAD 
Committee  concluded  review  of  the  service 
comments  and  promulgated  a Determination 
and  Findings**  that: 

• Designated  the  development  of  a joint 
requirement. 

• Required  the  generation  of  a joint  serv- 
ice operational  requirement  (JSOR) 
within  60  working  days. 


*Air  Munitions  Requirements  and  Development  Itr  dated 
24  April  73. 

**Air  Munitions  Requirements  and  Development  Itr  dated 
1 Aug  73. 


• Made  the  Navy  responsible  for  prepara- 
tion and  coordination  of  the  JSOR. 

• Authorized  both  service-advanced  de- 
velopment efforts  to  continue  through 
fiscal  year  1974. 

The  requirement  to  complete  the  JSOR 
within  60  working  days  proved  to  be  too 
short  a time  for  accomplishment.  Although 
the  Service  Secretary  was  the  action  addressee, 
the  performance  of  work  occurs  much  lower 
in  the  chain  of  command  and  must  be  of- 
ficially delegated.  Typically,  the  necessary 
time  for  delegation  was  not  given  considera- 
tion. In  this  case,  the  action  message  was  re- 
ceived by  the  Naval  Weapons  Center  (NWC) 
31  days  after  the  AMRAD  Committee  had  set 
a deadline  for  the  finally  coordinated  JSOR. 
Almost  half  the  available  time  was  taken  to 
get  the  action;  and  more  time  was  required  to 
implement  the  process.  Almost  no  time  was 
left  to  do  the  work. 

The  mechanism  used  for  joint  service  op- 
erational requirement  coordination  was  the 
Working  Party  for  Fuel  Air  Explosives  of  the 
Joint  Technical  Coordinating  Group  for  Air- 
Launched  Non-Nuclear  Ordnance  (JTCG/ 
ALNNO).  Use  of  this  group  offered  a pre- 
viously estabUshed,  informal  chain  of  com- 
mand prior  to  final  submission  of  the  program 
to  the  Chief  of  Naval  Operations  (CNO)  and 
to  the  Air  Force  Staff.  A final  draft  was  gen- 
erated and  submitted  to  the  Naval  Air  Sys- 
tems Command  (NAVAIR)  on  27  September 
1973.  The  NAVAIR  endorsed  the  draft  and 
submitted  it  to  CNO  on  19  October.  Ten 
days  later,  the  Chief  of  Naval  Operations  ap- 
proved the  draft  and  submitted  it  to  Air  Staff 
Department  of  the  Army,  and  Marine  Corps 
Headquarters,  along  with  the  request  that  a 
reply  be  provided  by  2 November  1973.  The 
final  joint  service  operational  requirement 
was  approved  and  submitted  to  the  AMRAD 
Committee  on  26  November. 

This  schedule  was  possible  only  because 
action  was  based  on  verbal  authorization  long 
before  formal  authorization  was  provided.  In 
this  case,  all  parties  were  cooperative.  In  other 
instances,  progress  has  been  stopped  because 
a task  could  not  be  assumed  until  after  it  had 
been  given  its  official  sanction. 
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As  the  draft  joint  service  operational  re- 
quirement went  through  the  review  process, 
several  significant  changes  were  made.  The 
changes  included: 

• The  one  1 ,000-pound  class  weapon  was 
changed  to  specify  one  500-pound  and 
one  2,000-pound  class  weapon. 

• Addition  of  a requirement  for  a foliage- 
discriminating  fuze. 

• Addition  of  a desired,  but  not  required, 
capability  for  supersonic  delivery  and  a 
150-knot  arming  speed. 

• Addition  of  a requirement  for  an  un- 
specified guidance  compatibility  in  the 
2,000-pound  weapon. 

The  results  of  these  changes  were  signifi- 
cant. The  development  of  two  versions  of 
weapon  instead  of  one  would  significantly  in- 
crease the  development  cost.  The  foliage  dis- 
crimination fuze  requirement  imposed  a po- 
tentially significant  fuze  cost  increase.  The 
desired  supersonic  release  capability  and  the 
150-knot  minimum  arming  speed  represented 
a long-standing  difference  in  opinion  between 
the  services  that  has  yet  to  be  resolved. 

The  AMRAD  Committee,  on  3 January 
1974,  promulgated  a Standardization/Rec- 
ommendation that: 

• Approved  the  JSOR. 

• Reaffirmed  a joint  requirement  (joint 
development  and  procurement  implied). 

• Designated  the  Navy  as  lead  service  with 
the  Air  Force  as  participant  and  the 
Army  as  monitor. 

• Established  that  joint  test  and  evaluation 
be  conducted  under  the  monitorship  of 
ODDR&E. 

• Established  a joint  management  struc- 
ture for  positive  resolution  of  critical 
technical  issues. 

• Required  a Joint  Development  Plan 
(JDP)  in  90  working  days. 


CREATION  OF  A JOINT 
DEVELOPMENT  PLAN 

On  3 January  1974,  the  Director  of  De- 
fense Research  and  Engineering  approved  the 
FAE  II  joint  service  project  and  requested  the 
Secretary  of  the  Navy  to  prepare  a Joint  De- 
velopment Plan  (JDP)  in  90  working  days. 
The  Naval  Weapons  Center  received  the  action 
to  prepare  the  JDP  on  1 Eebruary.  Assistance 
was  requested  immediately  from  the  Arma- 
ment Development  and  Test  Center. 

The  Chairman  of  the  JTCG/ALNNO  (now 
designated  Joint  Technical  Coordinating 
Group  for  Munitions  Development,  or  JTCG/ 
MD),  on  1 February  1974,  requested  that  the 
Joint  Development  Plan  be  written  by  the 
Working  Party  for  FAE.  The  previous  day,  the 
Joint  Logistics  Commanders  had  directed* 
that  joint  service  plans  and  programs  be  pre- 
pared by  the  JTCG  organization.  This  re- 
quirement did  not  create  a problem  in  that 
the  key  Air  Force  and  Navy  people  in  the 
FAE  II  project  were  members  of  the  Joint 
Technical  Coordination  Group. 

The  Joint  Development  Plan  was  written 
and  rewritten  many  times  during  the  period 
from  February  through  July  1 974.  Originally, 
the  plan  was  to  include  only  the  small  weapon. 
The  later  requirement  to  develop  both  weap- 
ons concurrently  created  a major  perturba- 
tion, primarily  from  a costing  viewpoint. 

Only  one  joint  service  project,  the  Gator 
project,  had  previously  been  planned  and  ap- 
proved under  the  current  guidelines.  The 
Gator  plan  was  recommended  as  a guideline. 
The  following  sections  of  the  plan  were 
among  those  required: 

Project  Summary /Authorizations 

Intelligence/Threat 

Operations 

Program  Management 


♦Joint  Logistic  Commanders  Itr  to  the  Chief  of  Naval  Op- 
erations, the  Chief  of  Staff,  Ait  Force,  and  the  Chief  of  Staff, 
Army,  dated  30  Jan  74. 
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Funding  and  Schedules 
System  Engineering 
Test  and  Evaluation 
Facilities 
Logistics 

Human  Factors  Engineering/Personnel 

Training/Support 

Security 

Applicable  MIL-STD/Specifications 

Environmental  Impact 

A Joint  Development  Plan  it  appears,  is 
usually  viewed  as  a necessarily  all-encompass- 
ing document.  In  this  document,  every  con- 
ceivable factor  has  been  considered,  planned, 
debated,  negotiated,  and  included.  The  result 
is  a wordy  document  that  is  read  in  detail 
only  by  the  negotiator/writer.  The  document 
is  costly,  bulky,  and  time-consuming  to  pre- 
pare. At  this  stage  of  a development  program, 
it  is  realistic  to  include  all  pertinent  factors, 
but  with  emphasis  placed  on  the  areas  that 
are  important  in  the  early  phases  of  the 
program. 

The  planning  for  the  FAE  II  program  might 
have  been  smoother  if  mention  only  of  cer- 
tain specialty  areas  had  been  made  just  to  in- 
dicate recognition  of  the  importance  of  the 
area  and  to  identify  the  time  frame  when  a 
complete  plan  would  be  generated.  When  this 
type  plan  was  attempted,  it  became  obvious 
that  each  specialty  group  felt  that  a complete 
plan  should  be  prepared,  and  every  conceiv- 
able detail  specified.  This  special  interest  pres- 
sure was  resisted.  The  sections  of  the  plan 
concerned  with  Facilities,  Logistics,  Human 
Factors  Engineering/Personnel,  and  Training/ 
Support  were  substantially,  though  not  suf- 
ficiently, reduced.  The  threat  always  existed 
that  any  specialty  group  resisting  supposed 
underemphasis  of  that  particular  specialty 
could  jeopardize  the  desired  schedule,  if  the 
group  were  so  inclined. 


In  retrospect,  it  appears  that  the  section  on 
Intelligence/Threat  should  have  been  deleted 
from  the  FAE  II  Joint  Development  Plan  in 
its  entirety.  The  original  entry  by  the  intel- 
ligence group  was  a 29-page  document  con- 
taining very  important  and  interesting  target 
information  but  this  information  had  no 
value  to  the  readers  of  a development  plan. 
Also,  this  entry  would  have  raised  the  secu- 
rity classification  of  the  Joint  Development 
Plan,  thus  causing  obvious  difficulties.  The 
intelligence  information  is  more  appropriate 
to  the  Requirement  Document. 

The  Joint  Development  Plan  should  be 
written  as  a working  document  and  reflect 
the  maturity  of  the  development  stage.  Key 
issues  and  planned  methods  of  resolution 
should  be  addressed.  As  the  program  pro- 
gresses, additional  sections  should  be  ex- 
panded, as  necessary,  to  insure  full  consid- 
eration. Attempts  to  make  the  JDP  all- 
encompassing  Horn  the  outset  not  only 
wastes  time  but  could  create  a document 
that  loses  relevance  with  time. 

Considerable  attention  was  devoted  to  the 
drafting  of  an  acceptable,  and  workable, 
management  structure.  At  the  time  of  Joint 
Development  Plan  approval,  the  FAE  II 
Project  Manager  was  located  at  NWC  and  was 
acting  for  NAVAIR  within  the  framework  of 
the  approved  JDP  and  other  controlling  Navy 
regulations.  The  relation  between  NWC  and 
the  Armament  Division  of  NAVAIR  that 
this  arrangement  reflected  had  been  estab- 
lished for  a long  period  of  time.  The  partici- 
pants had  demonstrated  the  ability  to  work 
with  efficiency  and  success.  At  the  time  of  a 
JDP  update  in  1975,  a change  in  location  of 
the  Project  Manager  Office  was  directed  by 
NAVAIR.  This  direction  was  brought  about 
by  a reorganization  within  NAVAIR,  and 
AIR  532  was  designated  Program  Manage- 
ment Office  for  all  conventional  munitions. 
The  effect  on  the  previously  established 
II  Project  Office  at  Naval  Weapons 
Center  was  a name  change.  The  Project  Office 
became  the  Project  Team;  the  NWC  Project 
Manager  was  redesignated  the  NWC  Project 
Team  Leader.  The  Air  Force  Deputy  Project 
Manager  remained  with  the  NAVAIR  Pro- 
gram Office,  but  was  Ibcated  at  NWC  until 
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his  reassignment.  A new  Air  Force  Deputy 
Project  Manager  was  recently  assigned  but  is 
located  at  Eglin  Air  Force  Base. 

The  Deputy  Program  Manager  position  was 
established  to  collocate  at  the  Naval  Weapons 
Center  with  the  Program  Manager.  The  Air 
Force  and  Navy  had  different  perceptions  of 
the  role  of  the  Deputy.  The  Navy  viewed  the 
occupant  of  this  position  as  a Deputy  who 
would  in  fact,  act  for  the  Program  Manager 
in  his  absence.  This  Deputy’s  primary  duties, 
though,  (as  directed  by  the  Air  Force)  were 
to  interface  with  the  Air  Force  to  ensure 
timely  satisfaction  of  Air  Force  requirements. 
The  Navy  assumed  that  the  Deputy  would  re- 
port to  the  Navy  Program  Manager  in  the 
same  manner  as  other  members  of  his  staff.  In 
fact  the  Air  Force  never  intended  that  the 
Deputy  report  to,  and  work  for,  the  Navy 
FAE  Project  Team  at  the  Naval  Weapons 
Center.  The  Air  Force  agreed  with  the  Navy 
in  that  the  Deputy’s  primary  purpose  at  the 
NWC  Program  Office  was  to  assure  USAF 
requirements  were  met.  The  Deputy  was 
authorized  direct  official  contact  with  his 
parent  organization  at  Eglin  Air  Force  Base. 
Copies  of  all  correspondence  were  made 
available  to  the  Navy  Program  Office.  The 
unique  difficulties  this  situation  generated 
arose  from  the  fact  that  a competitive  devel- 
opment was  still  underway.  Transmission  of 
NWC  test  results  to  Eglin  Air  Force  Base 
often  resulted  in  the  generation  of  differing 
conclusions  drawn  from  these  results.  This 
situation  was  viewed  as  a serious  problem  by 
the  Naval  Weapons  Center  FAE  Team.  The 
correspondence  issue  was  formally  resolved, 
and  the  autonomy  of  the  USAF  Deputy  was 
reaffirmed.  The  Air  Force  Deputy  Program 
Manager  is  now  free  to  convey  information 
to  pertinent  Air  Force  offices  concerning  any 
program  matter  deemed  appropriate.  In  like 
cases,  where  .major  impact  on  the  program 
could  result,  a Joint  Service  position  should 
be  sought. 

PANELS  FOR  RESOLUTION 
OF  ISSUES 

From  the  onset  it  was  recognized  that  dif- 
ferences of  opinion  between  the  services 


existed  and  that  ways  of  resolving  these  dif- 
ferences must  be  established.  By  AMRAD 
direction,  two  advisory  groups  were  estab- 
lished: the  Joint  Development  Review  Panel 
and  the  Joint  Technical  Review  Panel.  The 
AMRAD  intended  that  the  development  pro- 
gram have  joint  service  provision  for  address- 
ing technical  and  managerial  problems. 

The  Joint  Development  Review  Panel  con- 
sists of  equal  numbers  of  representatives  from 
NAVAIR  and  the  AFSC;  the  Panel  is  chaired 
by  a Navy  representative.  The  group  is  con- 
vened upon  formal  request  from  either  the 
Navy  or  the  Air  Force.  A unanimous  decision 
by  the  Panel  allows  the  Project  Manager  to 
take  direct  action  on  the  matter  under  dis- 
cussion. Nonconcurrence  after  deliberation 
calls  for  resolution  by  higher  authority 
through  the  existing  chain  of  command. 

The  Joint  Technical  Review  Panel  was  es- 
tablished to  assist  in  resolving  technical  issues 
that  predate  the  establishment  of  final  design 
baselines  for  full-scale  development.  This 
Panel  consists  of  four  representatives  each 
from  the  Naval  Weapons  Center  and  the 
Armament  Development  Test  Center  and  is 
chaired  by  the  senior  Naval  Weapons  Center 
representative. 

RESOLUTION  OF  BASELINE 
DESIGN 

Although  neither  the  Navy  nor  the  Air 
Force  had  demonstrated  success  by  the  end  of 
fiscal  year  1974,  each  service  had  flight  test 
assets.  Advanced  development  was  continued 
through  fiscal  year  1975.  However,  as  a result 
of  lead  service  designation,  the  Air  Force  ter- 
minated fiscal  year  1975  contracts,  reduced 
the  scope  of  advanced  development  testing, 
and  disbanded  the  Air  Force  FAE  II  team. 

The  Navy  assumed  that  they  would  be  re- 
sponsible for  selection  of  the  best  approach, 
after  a review  of  the  Navy  and  Air  Force  pro- 
grams. The  Air  Force  took  the  position  that 
this  was  a joint  service  function  to  be  per- 
formed by  the  Joint  Technical  Review  Panel. 
The  first  meeting  of  the  Panel  was  conducted 
in  April  1975  and  the  fiscal  year  1975  ad- 
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vanced  development  programs  of  both  serv- 
ices were  reviewed.  This  review  had  the  goal 
of  establishing  a baseline  design  through  reso- 
lution of  service  differences  and  identifica- 
tion of  the  best  approach  to  the  program. 
After  3 days  of  deliberation,  the  Panel  con- 
cluded that  additional  advanced  development 
Was  necessary  to  overcome  performance  limi- 
tations revealed  through  the  test  programs 
that  had  been  performed  by  each  service.  The 
Panel  specified  performance  goals  for  the  ad- 
vanced development  tests,  identified  key 
questions  requiring  resolution,  and  recom- 
mended design  changes  in  the  advanced  de- 
velopment models. 

At  this  review  the  Air  Force  insisted  that 
the  Navy  also  complete  advanced  develop- 
ment previously  terminated  by  the  Air  Force. 
After  unsuccessful  Navy  negotiations,  an  Ad- 
vanced Development  Plan  was  prepared  to  in- 
clude the  test  of  Air  Force  concepts.  This 
responsibility  was  viewed  as  a serious  problem 
by  the  performing  Navy  organization.  The 
following  excerpt  from  a memorandum  for 
record  expresses  the  Navy  feeling: 

“It  is  obvious  that  the  selection  of  the  FAE  II 
baseline  design  for  Engineering  Development  is 
thwarted  by  unintentional  internal  biases  within 
each  service. 

The  Naval  Weapons  Center  believes  that  the 
Navy  approach  provides  the  more  optimum  so- 
lution to  the  FAE  II  problem.  The  idea  of  the 
Navy  continuing  the  development  of  both  ap- 
proaches until  a selection  is  made  is  destined  to 
failure  no  matter  how  well  we  perform  because 
any  failure  is  likely  to  be  viewed  as  a fault  caused 
by  the  Navy.  It  is  therefore  recommended  that 
the  Navy  provide  a substantial  sum  of  money  to 
the  Air  Force  ...  to  show  good  faith,  and  if  the 
Air  Force  really  believes  that  their  approach  is 
superior  they  can  provide  any  amount  necessary 
to  augment  that  funding  necessary  to  demon- 
strating the  system  ...” 

This  approach  was  not  taken  because  of  fund- 
ing problems  and  because  the  Air  Force  felt 
the  responsibility  for  testing  candidate  ap- 
proaches should  be  with  the  lead  service. 

The  eventual  result  of  completing  the  ad- 
vanced development  plan  was  unanimous 


agreement  by  the  Joint  Technical  Review 
Panel  on  a baseline  design. 

OBSERVATIONS 

A joint  service  development  is  often  in- 
terpreted as  a development  of  an  item  by  two 
or  more  services,  each  of  which  has  significant 
influence  on  all  aspects  of  the  development. 
Such  an  interpretation  allows  dialogue  not 
only  on  the  basic  requirement  but  on  other 
areas  such  as  determination  of  the  best  ap- 
proach to  the  job,  the  materials,  contractor 
selection,  and  contract  monitoring.  Such  a 
dialogue,  while  it  can  be  beneficial,  can  also 
cause  endless  argument  on  minor  matters  and 
lead  to  potential  schedule  slippage  and  higher 
costs. 

The  FAE  II  joint  service  involvement 
brought  together  two  strong  technical  teams 
that  had  different  opinions  regarding  the  best 
approach  to  the  programs  and  different  opin- 
ions about  the  criteria  for  demonstrating 
weapon  feasibility.  The  resolution  of  these 
differences  resulted  in  the  program  entering 
engineering  development  2 years  later  than 
had  originally  been  proposed  by  the  Navy. 
The  question  of  how  much,  if  any,  time  and 
funds  were  wasted  during  the  2 year  period 
is  probably  impossible  to  answer.  Real  prob- 
lems were  solved  in  advanced  development- 
it  is  difficult  to  imagine  how  the  cost  of  solv- 
ing the  problems  could  have  been  reduced 
had  the  program  been  in  the  engineering 
development  phase.  Further,  the  technical 
risk  was  judged  acceptable  to  both  services. 

Experience  with  FAE  II  suggests  a number 
of  recommendations  for  consideration  in 
future  joint  service  developments; 

1.  Factors  considered  when  designating  the 
lead  service  should  include  an  assessment  of 
the  technology  and  demonstrated  perform- 
ance of  the  proposed  development.  (In  the 
case  of  FAE  II,  productive  interservice  com- 
petition promoted  technology  advances.  Lead 
service  designation  was  made  prior  to  the 
demonstration  of  a functional  model.  As  a 
result.  Air  Force  priorities  in  FAE  were  re- 
duced and  funding  reductions  forced  termina- 
tion of  advanced  development.  Unfinished 
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efforts  were  later  completed  by  the  Navy  at 
increased  cost.) 

2.  The  terms,  such  as  feasibility  demon- 
stration and  concept  validation,  often  used  to 
describe  the  outcomes  of  advanced  develop- 
ment (validation  phase),  should  be  described 
by  the  participating  services  in  quantitative 
terms  as  a prerequisite  to  the  initiation  of  en- 
gineering development.  This  was  not  done 
early  on  in  the  FAE  II  program  but  was  done 
eventually  at  a time  when  hardware  assets 
were  minimal  and  at  the  risk  of  rejecting  the 
best  approach  because  of  nonsystem  related 
test  failure  data. 

3.  Evaluate  with  care  the  developmental  ap- 
proach for  a program  that  appears  to  be  the 
optimum  approach,  whether  it  be  a joint  de- 
velopment or  a development  for  joint  service 
use.  Participating  services,  in  the  latter  ap- 
proach, would  be  concerned  with  satisfaction 
of  the  requirements  (including  cost,  perform- 
ance and  schedules),  but  would  not  be  active 
on  other  factors  in  day-to-day  management  of 
the  program.  Exceptions  to  this  would  occur 
in  cases  where  the  participating  service  was 
requested  to  assist  because  of  a special  com- 
petency. In  essence,  the  lead  service  would  be 
told  the  requirements,  but  not  how  to  accom- 
plish them. 

4.  Establish  a simple  agreement  relating  to 
issue  resolution  between  services  on  joint  pro- 
grams. Such  agreement  could  specify: 

a.  All  issues  deviating  from  agreed-upon 
plans  and  requirements  must  be  jointly 
resolved. 

b.  All  other  issues  can  be  unilaterally 
resolved. 

Suggestions  from  the  participating  service 
should  be  welcomed.  The  degree  of  attention 
to  be  devoted  to  these  inputs  should  be  de- 
termined by  the  lead  service. 


5.  Give  full  attention  to  the  designation, 
physical  location,  duties,  and  responsibilities 
of  the  Deputy  Program  Manager.  Provide  pro- 
cedures for  information  transmittal.  Consider 
the  size  of  the  project  before  assigning  a full- 
time Deputy  at  the  lead  service  field  activity. 
Such  collocation  may  not  be  warranted. 

6.  Simplify  the  Joint  Development  Plan  by 
deleting  detailed  planning  factors  that  will 
not  be  relevant  in  early  stages  of  the  program. 
Such  information  should  be  provided,  as  it 
becomes  relevant  in  subsequent  revisions 
when  the  item  is  more  firmly  developed.  State 
only  a framework  for  detailed  support  plans 
in  the  initial  JDP.  Detailed  plans  in  areas  such 
as  training,  quality  assurance,  maintainability, 
value  engineering,  configuration  management, 
data  management,  production  facilities,  and 
logistics  are  not  warranted  in  an  initial  Joint 
Development  Plan.  The  framework  should 
state  when  detailed  plans  for  such  areas  will 
be  provided.  Delete  the  currently  required 
section  on  Intelligence/Threat  in  the  JDP; 
this  information  is  better  placed  in  the  Re- 
quirement Document. 

7.  If  individual  Services  intend  to  compete 
for  the  lead,  a schedule,  mutually  agreed 
upon,  should  be  established  for  the  competi- 
tion by  the  competing  Services.  This  action 
would  allow  competition  completion  and  a 
review  of  the  results  prior  to  establishment  of 
the  lead  Service. 

8.  Recognize  that  most  less-than-major  pro- 
grams cannot  afford  the  formal  requirements 
necessary  for  the  major  programs. 

9.  Recognize  that  the  Joint  Development 
Plan  is  usually  written  long  before  a baseline 
design  is  established,  and  that  cost  estimates 
are  uncertain. 

10.  Establish  a corporate  memory  on  joint 
programs  to  allow  new  programs  to  benefit 
from  previous  programs.  □ 
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Mr.  Janies  A.  Bowen  is 
Head,  Weapons  Systems 
Branch,  Naval  Weapons  Cen- 
ter (NWC),  China  Lake,  CA, 
where  he  is  responsible  for 
all  Navy  Fuel-Air  Explosive 
efforts.  Mr.  Bowen  began 
his  professional  career  at  the 
Naval  Weapons  Center  (then  NOTS)  in  1955.  His  early  ex- 
perience was  in  design  and  developmental  engineering  on 
warheads  and  solid  propellant  propulsion  systems.  He  dem- 
onstrated the  first  underwater  launch  of  missiles  by  means 
of  buoyant  encapsulation.  As  Head,  Astro  Propulsion  Branch, 
he  developed  the  first  prototype  missile  fluid  injection  thrust 
vector  control  system  that  was  later  used  on  the  second  stage 
of  the  POLARIS  A-3. 

Mr.  Bowen  became  Head  of  the  Weapons  Systems  Branch 
in  May  1966  and  assumed  responsibility  for  the  Navy  Fuel- 
Air  Explosive  (FAE)  technology  projects.  He  later  became 
Program  Manager  for  the  first  FAE  weapon  to  be  opera- 
tionally deployed  by  the  services. 

Mr.  Bowen  was  awarded  the  American  Ordnance  Associa- 
tion “Harvey  C.  Knowles”  Award  in  May  1972  for  a major 
technical  contribution  related  to  armament.  In  the  same  year 
he  received  the  Michelson  Laboratory  Fellow  in  Management 
Award  for  his  technical  and  administrative  performance  in 
developing  Fuel-Air  Explosive  technology.  Mr.  Bowen  re- 
ceived his  B.S.  Engineering  from  Texas  A&M  College,  in 
1955. 


Captain  Ronsdd  S.  Fry  is 
the  Air  Force  Deputy  Program 
Manager,  FAE  II  Joint  Service 
Program.  He  is  assigned  to  the 
Armament  Development  Test 
Center,  Deputy  for  Armament 
Development,  Munitions  Sys- 
tems Program  Office,  Eglin 
AFB,  FL.  Captain  Fry  has  had  more  than  7 years  research 
experience  in  fuel-air  explosive  technology. 

Captain  Fry  received  his  BSE  degree  in  1969,  his  MSE  de- 
gree in  1970,  and  his  Professional  Engineering  degree  in 
1975.  All  three  degrees  were  awarded  by  the  University  of 
Michigan.  In  graduate  school  he  specialized  in  combustion 
and  shock  wave  propagation  phenomena  directly  related  to 
the  FAE  problem.  Captain  Fry’s  initial  active  duty  assign- 
ment (1974)  was  as  a project  engineer  on  the  Air  Force  FAE 
technology  effort  discussed  in  this  article.  He  participated  in 
many  of  the  Joint  Technical  Review  Panel  meetings  that 
shaped  the  validation  phase  of  the  FAE  II  development. 
Shortly  following  program  transition  to  full  scale  develop- 
ment phase  Captain  Fry  was  assigned  to  his  current  position. 

Captain  Fry  is  author  of  numerous  technical  articles  on 
the  subject  of  FAE,  including  publications  in  the  American 
Institute  of  Aeronautics  and  Astronautics  Journal  and  the 
internationally  recognized  Acta  Astronautica  and  the  Fif- 
teenth Symposium  (International)  on  Combustion.  Captain 
Fry  graduated  from  the  Air  Force  Squadron  Officer  School, 
Class  77-B,  1977,  at  the  top  third  of  his  class. 
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Computer  System  Simulation: 

A Design  Evaluation  Tool 

by 

Major  Robert  Steven  Feingold,  USAF 


In  an  important  sense,  computer  system  simulation  studies  are  to  the  evaluation  of  com- 
puter system  design  what  wind  tunnel  tests  are  to  the  evaluation  of  air  vehicle  designs.  For 
years,  Program  Managers  responsible  for  aeronautical  system  developments  have  relied  on 
the  results  from  wind  tunnel  tests.  The  same  opportunities  for  evaluating  and  reducing  the 
development  risk  associated  with  embedded  computer  systems  are  now  available  to  Program 
Managers  through  the  use  of  computer  system  simulation. 


PART  I 

Introduction 

The  increasing  complexity  of  weapon  sys- 
tems is  ever  present  in  military  system 
procurement.  Complexity  affects  develop- 
ment costs,  development  schedules  and,  once 
the  system  is  deployed  for  use,  operational 
support  costs.  Complexity  has  a significant 
impact  on  user-stated  system  performance 
goals.  Generally,  as  design  complexity  in- 
creases, so  does  the  level  of  effort  required  to 
meet  user  needs— on  time  and  within  budget. 

Among  the  most  complex  weapon  systems 
are  those  that  contain  embedded  computers. 
An  embedded  computer  system  is  a collection 
of  hardware  and  software  that  serves  to  sup- 
port an  overall  weapon  system  mission.  Em- 
bedded computers  are  used  to  process  radar 
track  inputs,  drive  display  devices,  control 
propulsion  systems,  and  supervise  the  launch- 
ing of  various  categories  of  ordnance.  Com- 
puters are  employed  directly  by  combat  ele- 
ments to  process  intelligence  data,  to  route 


command  message  traffic,  to  solve  complex 
trajectory  problems,  and  to  assist  in  con- 
trolling the  movement  of  supplies  and  re- 
placement parts  to  front  line  units.  Because 
computers  can  solve  mathematically-stated 
problems  rapidly,  embedded  computer  sys- 
tems will  continue  to  exert  great  influence 
on  weapon  system  design  decisions  and  on 
weapon  system  complexity. 

The  development  process  for  computer 
hardware  is  well  understood  and  for  the  most 
part  is  practiced  with  success:  The  same  can- 
not be  said  for  computer  software.  Of  several 
embedded  computer  system  developments 
that  have  experienced  problems,  the  majority 
of  the  problems  can  be  attributed  to  software 
development  management  issues. 

Like  hardware,  computer  software  goes 
through  concept,  validation,  full-scale  devel- 
opment, and  production-deployment  phases. 
Figure  1 shows  this  development  cycle.  The 
major  concern  in  the  concept  phase  is  the 
derivation  of  the  computer  system  perform- 
ance and  support  requirements  in  consonance 
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Figure  1,  Software  Development  Cycle 


with  the  overall  mission  requirements. 

In  the  validation  phase  the  design  effort  is 
of  major  concern.  It  is  in  this  phase  that  pro- 
posals for  hardware  and  software  configura- 
tions are  brought  forward  and  evaluated 
against  user  requirements.  In  the  validation 
phase,  design  decisions  are  made  that  deter- 
mine how  well  the  computer  system  will  meet 
its  performance  goals. 

In  full-scale  development  concentration  is 
on  the  acquisition  of  hardware  and  the  pro- 
gramming of  software  in  accord  with  the  de- 
sign selected  in  the  validation  phase.  Full-scale 
development  is  a lengthy  phase  that  involves 
the  testing  of  hardware  and  software  in  a con- 
trolled environment. 

The  production-deployment  phase  starts 
the  operational  baseline  has  been  estab- 
lished. Software  can  be  reproduced  easily  for 
deployment.  Hardware  must  proceed  through 
the  usual  manufacturing  and  qualifying  steps. 

Although  the  development  cycle  follows 
the  traditional  approach,  the  potential  for 
encountering  major  software  development 
problems  remains  high.  Problems  frequently 
occur  as  the  result  of  decisions  made  early  in 
the  development  cycle.  Historically,  the  prin- 
cipal reason  given  for  hardware-software  de- 
velopment problems  is  the  low  level  of  man- 
agement  attention  given  embedded  computer 
systems  by  government  program  offices.  Of- 
ten the  effort  devoted  to  hardware-software 
design  trade-off  studies  (in  the  validation 
phase)  is  not  sufficient  to  identify  potential 
problems. 


One  reason  for  not  devoting  adequate  re- 
sources to  computer  system  hardware  and 
software  trade-off  studies  is  lack  of  appropri- 
ate tools.  Software  design  trade-offs  are  dif- 
ficult to  study  before  software  is  prepared  or 
before  hardware  becomes  available.  The  com- 
puter system  designer  often  makes  hardware 
configuration  and  software  implementation 
decisions  based  on  experience.  The  conse- 
quence often  is  an  imbalance  in  total  system 
performance.  The  embedded  computer  sys- 
tern  may  fail  to  support  the  weapon  at  the 
desired  level  of  effectiveness. 
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The  primary  goal  of  this  paper  is  to  pre- 
sent a computer  system  design  evaluation 
methodology  appropriate  for  the  validation 
phase.  The  methodology  involves  user,  de- 
signer, and  developer  in  an  iterative  and  in- 
teractive design  evaluation  activity.  The 
methodology  relies  on  the  operations  research 
technique  of  system  simulation.  Simulation 
programs  represent  relevant  system  and  sub- 
system relationships  in  model  form  and  can 
be  used  to  evaluate  the  cost,  schedule  arid 
performance  impact  of  proposed  designs  in 
response  to  user-stated  requirements. 

A “hypothetical  situation”  is  used  to  il- 
lustrate how  computer  system  simulation  can 
be  employed  in  the  acquisition  process.  This 
approach  is  intended  to  dramatize  the  process 
by  permitting  an  indepth  look  at  the  com- 
puter system  simulation  technique.  The  ob- 
jective is  to  show  how  this  technique  can  be 
used  by  a program  office  responsible  for  an 
embedded  computer  system  development. 


PART  II 


Computer  System  Design  Evaluation 
Methodology 


Before  delving  into  the  details  of  the  com- 
puter system  simulation  technology,  first 
examine  where  computer  system  simulation 
fits  into  a conventional  design  evaluation 
process. 

The  design  evaluation  process  discussed 
consists  of  six  steps,  related  as  shown  in  Fig- 
ure 2.  The  process  is  iterative.  The  design 
evaluation  team  can  use  the  results  from  ex- 
periments to  develop  additional  design 
choices  and  to  suggest  changes  to  existing 
user  requirements.  This  iterative  process,  how- 
ever, cannot  go  on  forever.  Time  is  usually  the 
major  constraint.  More  important,  there  is  a 
limit  to  the  sensitivity  that  model-based  eval- 
uation techniques  can  show,  given  gross 
statements  of  requirements  and  design  op- 
tions. The  evaluation  must  culminate  in  a set 
of  design  decisions  to  be  carried  forward  for 
implementation  into  hardware  and  software. 


Figure  2,  Design  Evaluation  Process 
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requirements  analysis-step  1 

An  understanding  of  the  user’s  objective 
is  basic  to  the  success  of  the  design  process. 
Often  achieving  this  understanding  is  difficult. 
The  path  the  designer  follows  frequently  is 
cluttered  with  user  jargon  and  user  design 
biases.  To  overcome  these  and  similar  prob- 
lems, I advocate  the  formation  of  user- 
designer-developer  teams  that  can  function 
throughout  the  entire  process.  Clarification 
of  user  intent  and  developer  capability  at  an 
early  stage  usually  contributes  to  overall 
success.  Such  clarification  reduces  the  num- 
ber of  false  starts  and  shortens  the  design  ap- 
proval process  later. 

The  output  from  Step  1 is  an  understand- 
ing of  the  problem,  some  insight  into  the  feas- 
ible set  of  solutions,  and,  most  important,  the 
estabhshment  of  an  integrated  user-designer- 
developer  team  that  can  function  effectively 
throughout  the  remaining  five  steps. 

DESIGN  ALTERNATIVE  SYNTHESIS-STEP  2 

Synthesis  of  system  design  alternatives  is  a 
creative  process  that  combines  the  designer’s 
understanding  of  the  requirement,  his  abili- 
ties, his  education  and  his  past  experience. 
Alternative  evaluation  takes  place  at  this  step 
in  the  process.  Although  many  proposals  will 
be  made,  only  a few  will  be  worth  further 
study.  Judgment  is  essential  to  eliminate  pro- 
posals that  do  not  merit  further  considera- 
tion. In  Step  2,  the  judgment  is  supplied  by 
user  representatives  working  in  conjunction 
with  the  system  design  and  development 
specialists. 

In  the  past,  design  evaluation  was  accom- 
plished using  techniques  that  did  not  always 
consider  the  complexity  of  modern  computer 
systems.  In  many  cases  this  could  not  be 
avoided  because  economical  evaluation  tech- 
niques  were  not  available  for  doing  indepth 
analyses,  nor  were  there  any  techniques  suf- 
ficiently responsive  to  the  designer’s  need  to 
provide  quick  and  accurate  answers  to  design 
trade-off  questions.  The  computer  system 
simulation  approach  is  both  available  and  re- 
sponsive. 


MODEL  BUILDING-STEP  3 

The  process  of  model  building  satisfies  two 
critical  requirements.  First,  model  building 
permits  the  user-designer-developer  team  to 
articulate  a combined  knowledge  of  require- 
ments and  design  alternatives  into  a physical 
entity.  The  model  is  understandable  and  can 
be  subjected  to  intensive  analysis. 

The  second  major  requirement  to  be  satis- 
fied during  model  building  is  the  development 
of  a responsive  tool  which  is  useful  through- 
out the  remainder  of  the  process.  The  model, 
if  constructed  properly,  extends  the  intel- 
lectual abilities  of  the  design  evaluation  team 
by  providing  an  insight  to  system  component 
relationships,  rather  than  only  masses  of  num- 
bers to  analyze.  The  design  team  can  try  com- 
binations of  design  and  requirement  variables 
to  arrive  at  an  understanding  of  what  the  pro- 
posed system  will  do  and  how  it  will  behave 
under  a variety  of  circumstances. 

A language  useful  for  building  models  for 
the  application  being  described  is  called  the 
Extendable  Computer  System  Simulator  II 
(ECSS  II).  This  language  was  developed  by 
the  RAND  Corporation  for  the  US  Govern- 
ment, under  sponsorship  of  the  Federal  Com- 
puter Performance  Evaluation  and  Simulation 
Center  (FEDSIM)  and  the  General  Services 
Administration.  The  ECSS  II  is  used  here  be- 
cause it  satisfies  the  two  modeling  require- 
ments. understandability  and  responsiveness. 
The  ECSS  II  has  an  English-like  syntax  and 
provisions  for  compact  descriptions  of  com- 
puter elements  (hardware  and  software).  The 
flexible  and  extendible  ECSS  II  can  be  modi- 
fied and  is  economical  to  run.  Report  genera- 
tion capabilities  and  data  collection  facilities 
are  most  acceptable. 


EXPERIMENTS-STEP4 

Experimental  runs  made  wi.h  a model 
should  be  designed  to  explore  as  much  of  the 
design  and  requirement  response  surface  as 
possible.  An  exhaustive  search  is  impossible. 
Experiments  should  be  performed  that  allow 
the  design  evaluation  team. 
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• to  validate  the  model,  and 

• examine  system  performance  behavior 
relative  to  changes  in  sensitive  design 
variables. 

Model  validation  is  a major  subject.  A 
model  should  be  validated  to  a point  where 
its  behavior  can  be  explained  by  either  a 
model  coding  error  or  a phenomenon  which 
the  modeler  can  reasonably  expect  from  the 
actual  system.  The  outputs  of  the  model  may 
be  considered  vahd  if  they  are  explainable 
without  massive  contortion  of  facts  or  logic. 

The  simulation  experiments  provide  nu- 
merical data  on  the  performance  of  a pro- 
posed system,  given  input  design  variables  and 
a simulated  operational  environment.  A series 
of  experiments  are  usually  run,  each  with 
logical  variations  made  to  the  input  param- 
eters. After  the  experimental  phase,  the  out- 
puts are  correlated  with  the  changes  in  design 
input.  Thus,  a better  understanding  of  system 
behavior  is  achieved. 

For  most  computer  system  simulation 
models,  there  are  three  classes  of  input  vari- 
ables: 

• Variables  that  represent  hardware  op- 
tions such  as  processor  speed,  memory 
size  and  input-output  capacity. 

• Variables  associated  with  software  design 
options.  In  this  class  are  various  possibili- 
ties for  implementing  application  soft- 
ware, memory  management  techniques, 
and  multiprogramming  control  schemes. 

• Variables  related  to  the  operational  en- 
vironment. For  example,  a message  han- 
dling system  operates  in  an  environment 
where  message  arrival  rates  and  sizes  are 
quantified,  and  specifications  for  mes- 
sage response  times  are  given. 

The  outputs  from  the  experiments  carry 

onto  the  fifth  step  of  the  process:  Evaluation 

of  simulation  results. 

RESULT  EVALUATION-STEP  5 

The  fifth  step  in  the  process  is  intended  to 


result  in  one  of  three  possible  decisions.  If 
the  results  of  the  experimental  runs  indicate 
that  the  proposed  designs  are  inadequate,  the 
team  must  go  back  to  Step  2 and  try  to  de- 
velop new  designs.  The  processes  in  Steps  3, 

4,  and  5 are  repeated.  This  situation  is  a com- 
mon occurrence.  Often  a system  design,  on 
paper,  appears  to  satisfy  the  requirement  but 
turns  out  to  be  a failure.  Usually  this  is  be- 
cause of  the  complex  resource  demand  pat- 
tern produced  by  embedded  computer  sys- 
tem software.  When  a simulator  conclusively 
shows  that  a problem  exists,  the  design  eval- 
uation team  must  try  to  develop  new  designs. 

The  second  possible  decision  is  closely  re- 
lated to  the  first  but  differs  enough  to  justify 
separate  treatment.  After  several  iterations, 
it  may  become  clear  to  the  design  evaluation 
team  that  the  state-of-the-art  does  not  provide 
a design  feasible  for  satisfying  the  require- 
ment. In  this  case,  the  design  evaluation  team 
might  challenge  the  requirement  by  using  the 
model  to  evaluate  the  effect  of  a relaxed  re- 
quirement on  the  design  choice.  Full  user  par- 
ticipation in  this  kind  of  exercise  is  manda- 
tory. Modifications  made  to  the  requirement 
inputs  of  the  model  are  for  experimental 
purposes,  to  determine  the  trade  off  of  sys- 
tem cost  to  changes  in  requirements. 

The  third  possible  decision  represents  the 
end  product  of  the  process  to  this  point:  the 
selection  of  a feasible  system  design.  Through 
use  of  the  model,  the  design  evaluation  team 
can  select  one  or  a small  number  of  designs 
worthy  of  further  consideration.  Given  such  a 
determination,  the  design  evaluation  team 
can  move  to  Step  6. 

FINDINGS  AND  DOCUMENTATION-STEP  6 

The  final  step,  although  it  may  appear 
anticlimactic,  is  important.  The  design  eval- 
uation team  must  report  findings.  The  find- 
ings become  the  basis  for  seeking  approval 
to  continue  work.  This  documentation  must 
be  clear  on  a number  of  key  points.  First, 
the  process  used  to  translate  the  requirement 
into  model  inputs  must  be  absolutely  clear. 
Second,  the  documentation  must  clearly  and 
completely  identify  all  of  the  assumptions 
for  each  of  the  three  classes  of  input  that 
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went  into  the  fabrication  of  the  model.  Third, 
the  model  must  be  described  so  all  the  pre- 
viously identified  assumptions  are  clearly 
discernible  within  the  model  representation. 
Fourth,  the  data  collected  from  each  experi- 
mental run  must  be  defined  so  that  an  ac- 
curate interpretation  of  numerical  values  can 
be  made.  Fifth,  the  experimental  design  must 
be  described  and  justified.  Finally,  the  results 
from  each  experimental  run  must  be  de- 
scribed in  terms  that  support  the  conclusions 
and  recommendations  of  the  design  evalua- 
tion team. 

These  documentation  requirements  may 
seem  excessive;  however,  computer  system 
simulation  is  viewed  by  some  as  suspect.  In 
the  past,  full  disclosures  of  methodology 
were  not  made,  and  it  was  not  apparent  how 
the  recommendations  were  derived.  There  is 
no  such  thing  as  “hard  simulation  data”  upon 
which  high  dollar  decisions  can  be  made  with- 
out some  risk. 


PART  III 

Illustration 

HYPOTHETICAL  REQUIREMENT 

To  illustrate  the  use  of  computer  system 
simulation  in  the  embedded  computer  system 
design  evaluation  process,  a hypothetical 
situation  (hypothetical  in  terms  of  the  num- 
bers developed  and  assumptions  made)  is  pre- 
sented. In  this  situation  the  first  two  steps 
of  the  design  evaluation  process  are  illus- 
trated; requirements  analysis  and  design 
alternative  synthesis.  The  basis  for  the  illus- 
tration is  a military  system  design  problem.  A 
typical  user  requirement  and  one  design  pro- 
posal is  presented.  Although  only  one  design 
proposal  is  discussed,  the  methodology  can 
handle  any  number  of  design  proposals. 

Assume  a tactical  field  activity  needs  an 
improved  capability  to  process  information 
on  potential  tactical  targets.  Next  assume  that 
the  activity  must  provide  responsive  reports 
to  air  component  commands  on  possible 
tactical  air  missions  that  should  be  flown 
against  the  targets.  Specifically,  the  field  ac- 


tivity is  to  search  lists  of  known  targets,  up- 
date these  data,  and  display  those  targets  that 
have  attributes  corresponding  to  the  input 
search  arguments. 

In  addition,  timely  printed  reports  that 
display  target  data  in  a variety  of  ways  are 
to  be  obtained.  Report  formats  must  be  flex- 
ible to  satisfy  new  requirements  encountered 
in  the  field. 

In  general  terms,  three  capabilities  are  re- 
quired; 

• An  ability  to  input  target  data  to  the 
system  for  retrieval  later; 

• An  ability  to  search  and  update  the  data 
base  for  records  satisfying  a number  of 
search  input  characteristics;  and 

• A facility  to  provide  printed  reports  of 
previously  conducted  data  base  searches. 

Also,  the  user  desires  to  record  all  system 
transactions  in  printed  form  so  that  hard 
copies  of  all  data  base  changes  and  accesses 
will  exist. 

Based  on  his  knowledge,  the  user  provides 
additional  details  of  his  requirement.  First,  he 
defines  a system  encounter  to  be  any  series  of 
interactions  between  a human  operator  and 
the  mechanized  system.  Example;  a typical 
data  input  encounter  consists  of  an  input 
stream  of  characters  representing  target  data. 
The  output  is  a signal  from  the  system  that 
the  target  data  are  safely  stored  in  the  data 
base.  Second,  the  user  provides  information 
on  the  number  of  encounters  that  the  system 
is  required  to  process  per  hour.  Third,  the 
user  provides  an  indication  of  the  number  of 
characters  of  input  associated  with  each  en- 
counter category.  Finally,  some  indications 
as  to  the  volume  of  print  characters  needed 
to  complete  each  encounter  are  provided. 

In  Table  1 the  general  requirements  are 
summarized  for  all  encounter  categories.  By 
themselves  these  requirements  are  not  suf- 
ficient to  develop  design  alternatives,  let 
alone  build  a simulation  model.  Assume  that 
additional  information  exists  that  helps  refine 


Vol.  I,  No.  4 


67 


the  user  environment  to  a detailed  level  and 
allows  a design  (and  model)  to  be  proposed. 
For  example,  it  is  safe  to  assume  that  each 
encounter  in  reality  is  a series  of  man-machine 
interactions  consisting  of  a human  stimulus 
input,  system  processing,  and  final  output  re- 
sponse. Further,  assume  that  the  system  op- 
erator interfaces  with  the  equipment  through 
a keyboard  cathode  ray  tube  (CRT)  display 
device.  By  implication,  the  operator  must  key 


Table  1,  General  User  Requirements 


En- 

counter 

category 

En- 
counters 
per  hour 

Input 

char- 

acters 

Output 

print 

characters 

Data  input 

30 

80-  1,000 

480  - 6,000 

Data  base 
search  and 
update 

12 

40  - 200 

4,800  - 24,000 

Report 

generation 

6 

30-45 

96,000-  144,000 

inputs  at  a given  rate.  At  a specified  reading 
speed,  the  outputs  provided  on  the  CRT  are 
to  be  examined.  The  operator  must  then 
mentally  review  the  actions  that  have  been 
taken  and  the  actions  to  be  taken  before  key- 
ing the  next  interaction  stimulus.  Finally, 
assume  that  the  data  base  is  stored  on  a 
random  access  device,  say  a disk,  that  is 
accessed  a number  of  times  to  satisfy  each 
man-machine  interaction. 

Table  2 provides  the  numberical  values 
associated  with  these  detailed  assumptions. 
Table  3 is  a list  of  those  qualitative  assump- 


tions needed  to  complete  the  background 
material. 

HYPOTHETICAL  DESIGN 

One  design  is  discussed  here  to  illustrate 
the  design  proposal  synthesis  process. 


Table  3,  Qualitative  Assumptions 


1.  Incoming  material  representing  an  encounter  (i.e., 
data  for  input,  search  argument  values,  etc.)  have 
interarrival  times  drawn  from  an  exponential  dis- 
tribution with  a known  mean. 

2.  Encounter  categories  are  of  equal  priority. 

3.  The  system  operator(s)  must  LOG  ON  to  the  sys- 
tem for  each  encounter  (to  satisfy  security  pro- 
cedures). 

4.  Every  stimulus  from  the  CRT  requires  a response 
(to  the  CRT). 

5.  All  encounters  are  logged.  Some  encounters  pro- 
duce a printed  output(s). 

6.  Response  time:  the  interval  from  arrival  of  encoun- 
ter material  to  time  last  line  of  output  is  printed. 

7.  The  system  should  be  compact,  consist  of  a 
minimum  of  parts,  be  reliable,  and  easily  main- 
tainable in  the  field.  The  technology  must  be  within 
the  state-of-the-art. 


Table  2,  Quantitative  Assumptions 


Encounter 

category 

Repetitions 

per 

encounter 

Average  key 
stroke  time 
(3  characters 
per  second) 

Response 
read  time 
(500  characters 
per  second) 

Think  time 
(seconds) 

Data  base 
accesses 

Data  input 

2-25 

26  - 333 

10-120 

10-125 

4-50 

Data  base  search 

2-10 

13-66 

5-24 

15-70 

100-  1,000 

and  update 

Report  generation 

2-3 

10-15 

4-6 

10-  15 

40  - 120 
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HARDWARE 

The  system  will  consist  of  a central  process- 
ing unit  that  has: 

• A memory 

• A multiplexor  interface  channel  to  the 
keyboard-CRT  devices  and  the  printer, 
and 


Device  characteristics  are  shown  with  each  sys- 
tem component.  The  devices  called  C.PATH, 
P.PATH  and  D.PATH  are  names  given  to 
entities  over  which  simulated  data  flows.  For 
example,  C.PATH  serves  the  cathode  ray  tube 
(CRT)  to  CHI  path  while  D.PATH  serves  the 
DISK  to  CH2  path. 


SOFTWARE 


• A burst  mode  interface  channel  to  the 
disk  devices  that  will  hold  the  data  base. 

Since  it  is  required  that  the  system  consist  of 
state-of-the-art  components,  the  capabilities 
of  each  subsystem  will  be  restricted. 

The  design  proposal  considered  here  is 
shown  in  Figure  3.  The  names  given  to  each 
class  of  components  appear  later  in  the  model. 


Hardware  in  a computer  system  is  of  no 
use  without  software  and  software  must  be 
designed  in  appropriate  detail.  The  following 
assumptions  simplify  the  illustration.  First, 
consideration  is  not  given  to  operating  sys- 
tem overhead.  The  assumption  is  made  that 
overhead  is  accounted  for  within  the  applica- 
tion software.  Second,  each  major  category 
of  input  is  served  by  a different  program  that 
must  be  in  the  central  processing  unit  (CPU) 


Cathode  ray  tube 


C.  PATH  entity  over  which  simulated  data 
from  the  cathode  ray  tube  flows 

P.  PATH  entity  over  which  simulated  data 
flows  to  the  printer 


Figure  3,  System  Design  Configuration 


D.  PATH  entity  over  which  simulated  data 
flows  to  disk 
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memory  for  the  interactions  between  man 
and  machine  to  take  place.  Third,  the  system 
supports  multiprogramming.  Many  programs 
can  reside  in  CPU  memory  at  one  time  and 
compete  for  the  CPU  and  other  system  re- 
sources. Finally,  there  is  a separate  program 
that  produces  a hard  copy  printout  (printed 
sheets  of  paper)  from  files  stored  on  disk. 
Under  this  arrangement  each  program  that 
produces  a hard  copy  printout  places  the  data 
on  disk  for  later  printing.  This  process  is 
called  spooUng  and  is  used  often  to  control 
access  to  printing  devices. 

Elaboration  is  required  for  each  of  the 
processing  categories.  The  actions  of  the  op- 
erator are  shown  in  Figure  4.  First,  the  op- 
erator receives  the  material  with  which  he 
must  work.  The  material  may  be  new  input 
data,  information  for  searches  or  updates,  or 
parameters  for  a required  printed  report. 
The  operator  LOGS  ON  and  waits  for  the  sys- 
tem to  indicate  access  to  the  necessary  system 
resources.  Next,  the  operator  performs  a 
series  of  machine  interactions  until  the  work 
is  complete.  The  operator  then  requests  the 
machine  to  produce  a printed  product.  Next 
the  operator  LOGS  OFF.  Because  it  is  likely 
that  additional  encounter  material  is  waiting, 
the  operator  will  resume  performance  of  a 
like  task. 

The  software  process  supporting  the  Data 
Input  encounter  category  is  shown  in  Figure  5. 
First,  the  LOG  ON  is  performed  and  a signal 
to  proceed  is  given  the  operator.  Each  time 
the  operator  provides  an  input,  the  program 
places  the  data  on  disk,  updates  the  data 
base  index,  and  displays  a response  on  the 
operator’s  CRT.  When  all  interactions  are 
complete,  the  LOG  OFF  process  is  performed. 

The  Data  Base  Search  and  Update  en- 
counter process  is  more  complex.  See  Figure 
6.  After  performing  the  LOG  ON  process, 
each  operator  input  is  scanned  and  used  to 
control  the  search  and  update  of  the  data 
base.  Once  again,  when  the  interactions  are 
complete,  a LOG  OFF  is  performed. 

Figure  7 is  a flowchart  of  the  Report  Gen- 
eration process.  The  program  interprets  op- 
erator inputs,  retrieves  data  from  the  data 
base,  organizes  these  data  into  the  appropri- 


Figure  4,  Operator  Behavior 
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operator 
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HI 
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input 


Scan  input 
characters 


Store  on 
designated 
disk  unit 


Update  data 
base  index 


Reply  to 
operator  CRT 


Yes 


Handle 
requested 
LOGOFF  process 


Figure  5,  Input  Target  Data  Process 


ate  report  format,  and  stores  the  report  on 
the  output  spool  file  for  later  printing. 

One  final  process  completes  the  software 
program  set.  The  Print  Spool  Program,  shown 
in  Figure  8,  produces  all  the  printed  products 
for  the  system.  The  program  is  assumed  to 
operate  in  a batch  mode,  or  as  a background 
process,  and  can  serve  a system  with  multiple 
printers.  This  design  feature  enables  the  pro- 
gram to  later  request  a printout  from  the  sys- 
tem that  prints,  one  by  one,  the  lines  con- 
tained in  the  output  spooling  files. 


INITIAL  TIMING  STUDY 

Initial  timing  estimates  to  determine  the  re- 
sponse time  characteristics  of  the  system  can 
now  be  made.  First  a few  assumptions  are 
made  about  instructions  executed  per  pro- 
gram as  well  as  input-output  requirements 
for  each  program.  The  assumed  instruction 
counts  are  given  in  Table  4,  These  numbers 
represent  our  informed  estimate  of  what  is 
required  to  accomplish  each  process.  Ex- 
perience and  knowledge  of  commonly  used 
algorithms  form  the  primary  basis  for  the  low 
and  high  estimates. 


Table  4,  Process  Instruction  Counts 
(instructor/encounter) 


Encounter 

category 

Low  estimate 

High  estimate 

Data  input 

19,980 

48,500 

Data  base  search 
and  update 

1,057,200 

10,156,200 

Report 

generation 

565,200 

842,800 

Table  5 shows  the  low  and  high  estimates 
for  input  and  output  times.  Within  a process 
there  is  only  one  input-output  (I/O)  function 
performed  at  a time.  The  I/O  is  never  over- 
lapped with  instruction  execution.  The  values 
in  Table  5 include  all  I/O  for  each  process. 
Apin,  I/O  associated  with  printing  is  included 
with  each  of  the  three  processes.  The  precise 
flavor  of  the  values  shown  in  Table  5 results 
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from  expressing  I/O  in  seconds  rather  than  in 
characters  per  encounter.  This  is  necessary  be- 
cause the  CRT,  DISK,  and  PRINTER  transfer 
characters  at  different  rates. 


Table  5,  Input-Output  Timing 


Encounter 

Category 

Low  time 
estimates 
(seconds/ 
encounter) 

High  time 
estimates 
(seconds/ 
encounter) 

Data  input 

3.194 

36.685 

Data  base  search 
and  update 

21.83 

81.485 

Report 

generation 

243.808 

368.925 

To  complete  the  timing  study,  account 
for  operator  think  time  that  is  not  overlapped 
with  any  machine  process.  This  accounting 
can  be  complex  but  can  be  simplified  by 
assuming  that  for  the  three  interactive  proc- 
esses, think  time  is  20  seconds,  20  seconds, 
and  15  seconds  respectively.  In  summary, 
the  average  timing  estimates  shown  in  Table  6 
are  obtained  by  finding  the  mean  of  the  low 
and  high  timing  estimates.  This  is  reasonable 
if  instruction  counts,  data  base  accesses,  and 
transaction  character  sizes  for  I/O  are  drawn 
from  uniform  distributions  and  if  the  low  and 
high  values  given  in  Tables  4 and  5 are  used  as 
parameters. 


Table  6,  Timing  Summary 
(average  seconds  per  encounter) 

Timing 

element 

Data 

input 

Data  base 
search 
and 
update 

Report 

generation 

Execution  time 

.342 

56.067 

7.0400 

(1000000/second) 

Input/output 

19.393 

51.657 

306.366 

time 

Think  time 

270.0 

120.0 

25.0 

Total 

290.281 

227.724 

338.406 

Average  response  time 

285.470 

72 


Defense  Systems  Management  Review 


Figure  7,  Report  Generation  Process 


Figure  8,  Output  Spooler  Process 


Analyses  as  performed  here  are  common, 
and  for  simple  systems  are  adequate.  This 
type  analysis  does  not  account  for  all  re- 
sources needed  to  process  an  encounter.  For 
example,  the  analysis  discussed  does  not  in- 
dicate that  the  design  value  of  130,000  bytes 
of  memory  is  adequate  for  the  combination 
of  processes.  Further,  time  could  be  lost  as 
each  process  waits  for  resources  assigned  to 
other  processes.  Queues  can  develop.  First, 
there  are  four  devices  on  CHI  that  can  proc- 
ess only  three  messages  concurrently.  Second, 
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there  is  one  CPU.  A maximum  of  four  pro- 
grams can  request  this  resource  at  any  given 
time.  Third,  the  DISK  devices  must  be  ac- 
cessed one  at  a time.  Each  of  four  programs 
requesting  the  CPU  can  have  DISK  I/O  re- 
quests. Finally,  there  is  only  one  printer,  yet 
three  processing  programs  can  be  generating 
print  outputs. 

Looking  at  the  design  as  a system  of  queues 
is  an  approach  for  analyzing  the  impact  of 
potential  resource  contention  problems.  A 
mathematical  solution  to  these  queueing 
models  often  requires  simplification  that 
could  result  in  assuming  away  the  problem. 
Simulation  is  required  because  the  interac- 
tions between  resources  required  and  re- 
sources available  are  highly  complex. 


PART  IV 

Modeling  and  Experimentation 

In  Part  I,  I stated  the  primary  purpose  of 
this  paper:  to  present  a computer  system  de- 
sign evaluation  methodology  appropriate  for 
the  validation  phase  of  an  embedded  com- 
puter system  development.  This  purpose  pre- 
cludes a long  treatise  on  modeling  languages, 
modeling  techniques,  and  simulation  strat- 
egies. The  characteristics  of  the  model  used  in 
this  project  are  delineated  in  the  study  paper 
on  which  this  article  is  based.* 

The  model  is  written  in  ECSS  II  and  con- 
sists of  four  sections.  Each  section  performs 
a descriptive  or  an  operative  function  in  the 
simulation.  Persons  desiring  to  examine  the 
construction  of  the  model  will  be  interested 
in  the  basic  study. 

EXPERIMENTATION  PLAN 

Experimentation  can  be  an  extremely  com- 
plex process  even  with  a simple  simulation 


♦Robert  S.  Feingold,  “Computer  Design  Simulation:  A 
Design  Evaluation  Tool/’  Study  Report,  PMC  76-1,  Defense 
Systems  Management  College,  Fort  Belvoir,  VA  , 1976. 
(Available  from  NTIS  and  DDC,  Acquisition  No.  AD 
A026387) 


model.  Hence  ground  rules  were  established 
which  drastically  narrow  the  scope  of  ex- 
perimentation. First,  the  parameters  that  de- 
scribe the  requirement  are  kept  constant,  al- 
though the  simulation  program  permits  these 
variables  to  be  Input  at  model  execution  time. 

Table  7 contains  a complete  list  of  the  re- 
quirement parameters  and  the  values  are  given 
in  the  order  of  input  to  the  model.  In  addi- 
tion, certain  design  parameters  are  kept  con- 
stant. The  values  of  the  design  parameters 
are  given  in  Table  8. 

The  second  simplifying  ground  rule  states 
that  only  hardware  design  characteristics  such 
as  memory  capacity,  quantity  of  CRT  used, 
quantity  and  speed  of  printers,  and  the  cen- 
tral processing  unit  (CPU)  execution  rate  will 
be  varied  from  one  experimental  run  to  the 
next.  These  values  are  input  to  the  simulation 
through  the  SYSTEM  DESCRIPTION  section 
of  the  model.  Table  9 gives  the  range  of  values 
over  which  experiments  are  run. 


Table  7,  Baseline  Requirement  Parameters 


Parameter 

Data 

input 

function 

Data 

base 

search 

and 

update 

Report 

gen- 

eration 

1 . Stimulus  input 
Character  stream 
(Characters) 

40 

20 

15 

2.  Think  time 
(Seconds) 

20 

20 

15 

3.  Maximum  no. 
of  interactions 

25 

10 

3 

4.  Minimum  no. 
of  data  base 
accesses 

2 

50 

10 

5.  Maximum  no. 
of  data  base 
accesses 

2 

100 

20 

6.  Mean  encounter 
interarrival  time 
(seconds) 

120. 

300. 

600. 
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Table  8,  Baseline  Constant  Design  Parameters 


Parameter 

Value 

Length  of  simulation  run 

3600  seconds 

Data  input  program  core  size 

1 5000  bytes 

Data  base  search  and  update  program 
core  size 

60000  bytes 

Report  generation  program  core  size 

30000  bytes 

LISTOFF  program  core  size 

1 0000  bytes 

Data  base  record  size 

2000  bytes 

CRT  screen  capacity 

1 920  bytes 

Table  9,  Range  of  Major  Design  Parameter  Values 

Parameter 

Low 

High 

Number  of  CRT 

3 

8 

Instruction  execution  rate 
(ins/second) 

100000 

200000 

Memory  capacity  (bytes) 

130000 

260000 

Number  of  printers 

1 

2 

Printer  speed  (lines/minutes  — 
120  bytes/line) 

200 

600 

The  final  simplification  involves  the  per- 
formance measures  used  to  evaluate  each  ex- 
perimental run.  If  anything,  a simulation  pro- 
vides too  much  performance  data  and  often 


is  precise  to  the  fifth  decimal  position,  even 
though  its  basic  accuracy  may  be  question- 
able. The  principal  performance  measure  is 
the  average  response  time  for  each  encounter 
category.  Response  time  is  measured  in  sec- 
onds and  consists  of  two  components:  time 
spent  by  encounter  material  entities  in  a 
queue  prior  to  being  worked  on  by  a system 
operator  and,  time  spent  in  processing  the 
encounter  material  including  print  processing. 
The  total  average  response  time  is  reported 
for  each  category  along  with  the  average  total 
response  time  across  all  categories. 


Certain  other  performance  measures  must 
not  be  neglected.  These  measure  performance 
relative  to  the  utilization  and  queueing  char- 
acteristics for  the  various  computer  system 
resources.  Accordingly,  we  look  at  measures 
associated  with  the  CRT,  CPU,  PRINTER, 
PATH,  and  memory  devices. 


In  all,  twelve  experimental  runs  were  made 
as  part  of  the  study  on  which  this  paper  is 
based.  A partial  evaluation  followed  each  run 
to  determine  the  set  of  inputs  appropriate  for 
the  next  run.  As  stated  earlier,  experimental 
strategy  is  not  the  principal  topic  of  discus- 
sion, so  all  experimental  results  are  presented 
at  one  time. 

Experimental  Results 

To  begin,  look  at  the  results  from  the  base- 
line design.  The  initial  timing  study  predicted 
an  average  response  time  for  the  three  encoun- 
ter categories  of  290.281  seconds,  227.724 
seconds,  and  338.426  seconds,  respectively. 
With  this  in  mind.  Table  10  reveals  a startling 
result:  a response  time  of  1640.0  seconds. 

Why  such  results?  To  answer  this  question 
look  at  the  detail  performance  measures 
given  in  Table  11.  First,  waiting  time  in  the 
encounter  material  queue  is  high  at  1086.8 
seconds.  Given  that  the  average  response  time 
is  1640.0  seconds,  the  average  wait  time  in 
the  encounter  material  queue  accounts  for 
more  than  66  percent  of  this  figure.  Second, 
the  table  shows  that  CRT  utilization  is  almost 
100  percent  and  the  central  processing  unit 
is  busy  over  three-quarters  of  the  sampling 
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Table  10,  Baseline  Design:  Simulation  Results 
Response  Times  (Seconds) 


Table  1 1 , Baseline  Design:  Simulation  Results 
Performance  Measures 


Measure 

Data 

input 

function 

Data 

base 

search 

and 

update 

Report 

gen- 

eration 

Queue  time 

1339.1 

492.6 

352.7 

Internal  time 

481.6 

1357.0 

897.1 

Total  time 

1820.7 

1849.6 

1249.8 

Initial  prediction 

290.3 

227.7 

338.4 

Average  response  time 

1640.0 

Predicted  average  response  time 

285.5 

CRT  = 3 

KIPS  = 100 

MEMORY  = 130000 
PRINTERS  = 1/200  LPM 


Measu  re 

Value 

CRT  utilization  (percent) 

99.3 

Encounter  material  queue  (EMQ) 
length 

19.0 

Wait  time  in  EMQ 

1086.8 

CPU  utilization  (percent) 

75.8 

CPU  queue  (CPUQ)  length 

.7 

Percent  CPUQ  empty 

56.7 

Printer  queue  (PQ)  length 

.564 

Wait  time  in  PQ 

101.5 

Memory  utilization  (percent) 

77.7 

Memory  request  queue  (MRQ)  length 

.448 

Wait  time  in  MRQ 

37.5 

Table  12,  Experimental  Runs:  Simulation  Results— Performance  Measures 


EXPERIMENT 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

CRT,  quantity 

6 

6 

6 

6 

6 

6 

6 

6 

6 

6 

8 

KIPS 

100 

100 

100 

100 

150 

150 

150 

150 

150 

150 

200 

Memory  size 

130 

260 

130 

260 

130 

260 

130 

260 

130 

260 

260 

Printers,  quantity 

1 

1 

2 

2 

2 

2 

1 

1 

2 

2 

2 

Printer  speed 

200 

200 

200 

200 

200 

200 

600 

600 

600 

600 

600 

CRT  UTIL,  {percent) 

93.3 

90.7 

90.3 

93.8 

90.1 

93.3 

91.3 

92.6 

93.0 

93.8 

88.7 

EMQ  length 

5.2 

12.3 

8.4 

7.5 

10.5 

5.3 

7.7 

2.7 

6.6 

9.1 

2.93 

EMQ  wait  time  {seconds) 

296.3 

707.0 

481.7 

430.9 

603.1 

301.9 

442.0 

155.7 

379.6 

520.0 

167.4 

CPU  UTIL,  (percent) 

77.4 

90.0 

81.5 

85.8 

71.1 

80.6 

67.7 

72.8 

93.3 

70.5 

89.1 

CPUQ  length 

.73 

2.28 

.96 

2.57 

.51 

2.36 

.54 

1.69 

.60 

1.87 

4.09 

CPUQ  empty  (percent) 

53.4 

28.4 

47.3 

27.2 

63.4 

35.3 

64.1 

46.5 

62.0 

47.1 

19.1 

PQ  length 

3.16 

3.40 

.18 

.96 

.08 

1.25 

.21 

1.49 

.00 

.18 

1.50 

PQ  wait  time  (seconds) 

284.3 

408.2 

16.9 

86.2 

7.5 

88.3 

17.4 

105.8 

.09 

17.6 

110.5 

Memory  UTIL,  (percent) 

87.4 

65.4 

88.5 

69.4 

89.5 

69.6 

87.1 

73.0 

86.4 

60.2 

80.4 

MSQ  length 

1.96 

0.0 

2.36 

0.0 

2.07 

.09 

1.63 

.09 

2.24 

.11 

.27 

MRQ  wait  time  (seconds) 

89.1 

0.0 

106.2 

0.0 

91.1 

3.2 

63.2 

3.1 

94.0 

4.8 

9.7 

Average  response  time 

1039.6 

1581.6 

1115.0 

1024.8 

1184.4 

923.7 

939.2 

655.5 

927.0 

1017.1 

889.0 

EMQ  ° Encounter  material  queue 
CPUQ  = CPU  queue 
PQ  = Printer  queue 
MRQ  = Memory  request  queue 
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interval.  It  appears  that  the  system  needs 
more  CRT  devices  to  increase  the  level  of 
concurrent  processing. 

Apparently  the  initial  estimate  was  faulty, 
because  it  was  assumed  that  contention  for 
computer  system  resources  would  be  minimal. 
However,  contention  is  clearly  evident.  The 
data  shows  that  the  CPU  queue  (CPUQ)  is 
empty  56.7  percent  of  the  sampling  interval 
and  the  length  of  this  queue  averages  0.7,  a 
small  yet  significant  number.  Waiting  time 
for  memory  averaged  37.5  seconds  for  each 
memory  request.  This  delay  represents  ap- 
proximately 4 percent  of  the  average  internal 
processing  time.  A similar  analysis  using  the 
waiting  time  to  start  printing  figure  of  101.5 
seconds  yields  a percent  of  average  internal 
time  equal  to  11.1. 

That  more  CRT  are  required  is  clear.  Less 
clear,  but  apparent,  is  an  indication  that 
memory  capacity  should  be  increased  along 
with  the  CPU  execution  rate,  number  of 
printers  and  printer  speed.  But  how  much 
and  in  what  combinations? 

A complete  picture  can  be  made  of  the 
experiments  run,  showing  the  design  changes 
made  and  the  performance  measures  ob- 
tained. Table  12  provides  such  a picture.  Be- 
fore examining  the  numerical  values  for  the 
performance  measures  given  in  Table  12, 
note  the  experiments.  The  only  change 
made  to  the  baseline  to  get  Experiment  1 was 
to  double  the  number  of  CRT.  Next,  the 
memory  capacity  was  doubled  for  Experi- 
ment 2.  Experiments  3 and  4 repeat  Experi- 
ments 1 and  2 but  with  two  printers  instead 
of  one.  Experiments  5 and  6 repeat  Experi- 
ments 3 and  4 but  with  a CPU  execution  rate 
of  1 50,000  instructions  per  second  instead  of 
100,000.  Experiments  7,  8,  9,  and  10  all  have 
a CPU  execution  rate  of  150,000  instructions 
per  second,  a 600  line  per  minute  printer,  but 
are  otherwise  repeats  of  Experiments  1,  2,  3, 
and  4.  This  design  explores  that  portion  of 
the  performance  response  surface  which  can 
reasonably  be  considered  state-of-the-art  and 
responsive  to  the  requirement  given  earlier. 
Experiment  11  is  a run  to  see  the  results 
from  a design  possessing  abundant  system 
resources. 


What  do  these  data  tell  us  about  the  pro- 
posed design  options?  First,  there  is  an  in- 
teresting anomaly  between  Experiments  1 and 
2.  Memory  capacity  doubles.  Yet,  with  all 
other  design  parameters  held  constant,  aver- 
age response  time  increases . Close  examina- 
tion shows  that  in  fact  there  is  no  anomaly. 
With  more  jobs  in  memory,  the  CPUQ  is 
longer  and,  on  the  average,  the  CPUQ  is  less 
frequently  empty.  CPU  contention  is  much 
greater  in  Experiment  2 because  more  jobs 
can  be  in  memory  and  issue  a greater  num- 
ber of  requests  for  the  CPU  resource.  In  a 
like  fashion,  more  jobs  processed  means 
more  print  output  produced  in  a shorter 
time.  This  is  indicated  by  an  almost  two-fold 
increase  in  printer  queue  wait  time.  The  aver- 
age memory  queue  length  is  zero  which  in- 
dicates that  probably  260,000  bytes  of 
memory  were  not  needed. 

The  second  basic  observation  is  that  Ex- 
periment 8 yields  the  best  average  response 
time  of  655.5  seconds.  This  is  partially  ex- 
plained by  the  low  EMQ  wait  time  of  155.7 
seconds.  Further  explanation  is  provided  by 
observing  that  CPU  utilization,  relatively 
speaking,  is  low,  at  72.8  percent.  Memory 
utilization  is  similarly  low  at  73.0  percent. 
Taken  together,  this  indicates  that  overall 
system  resource  demands  are  best  balanced 
with  this  configuration.  On  a relative  basis, 
CPU  contention  is  lower,  allowing  individual 
jobs  to  proceed  to  an  input-output  operation 
and  thereby  release  the  CPU  resource  for 
other  jobs.  Progress  in  using  the  CPU  resource 
is  aided  considerably  by  the  1 50,000  instruc- 
tion per  second  execution  rate. 

Finally,  a major  observation  must  be  made. 
By  the  very  nature  of  the  experiments,  it  is 
impossible  to  know  what  performance  gains 
can  be  achieved  realistically  if  design  changes 
are  made  to  the  simulated  software.  For  now, 
we  see  that  there  is  a performance  limit  be- 
yond which  we  cannot  go,  given  the  nature 
of  the  experiments  used  in  the  illustration. 

In  summary,  it  was  shown  that  the  simula- 
tion model  could  be  used  to  evaluate  the  base- 
line configuration  proposed.  In  addition,  a 
series  of  experiments  can  be  run,  each  using  a 
slightly  altered  set  of  input  design  parameters. 
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Finally,  from  examination  of  the  performance 
output  measures,  it  is  possible  to  select  one 
or  more  design  options  that  exhibit  desirable 
performance  characteristics. 

PART  V 

Concluding  Remarks 

In  this  paper,  five  out  of  the  six  steps  out- 
lined in  Part  II  were  explained  and  illustrated. 
In  the  interest  of  brevity,  the  iterative  ele- 
ments of  the  procedures  and  the  documenta- 
tion requirements  were  not  stressed.  One  il- 
lustration does  not  conclusively  demonstrate 
the  usefulness  of  any  procedure  nor  does  it 
adequately  warn  potential  users  of  the  pit- 
falls  usually  encountered  during  design  eval- 
uation activity. 

The  model  developed  for  the  illustration 
can  be  extended  easily  to  examine  many 
software  design  implementations,  in  addition 
to  the  hardware  designs  shown  in  this  paper. 
New  models  appropriate  for  different  applica- 
tion areas  also  can  be  easily  written  using 
ECSS  II.  Some  examples  of  software  related 
implementations  that  might  be  addressed  by 
an  ECSS  II  model  are  listed  below: 

• Preempt-resume  CPU  dispatching  algo- 
rithms. 

• Priority  driven  job  scheduling. 

• Priority  driven  I/O  scheduling. 

• Program  segmentation  and  paging. 

• Program  overlay  structures. 

• Working  set  page  allocation  algorithms. 


• Double  buffering  of  I/O. 

• Multiprocessor  task  dispatching. 

• Dual  channel  disk  access  control  algo- 
rithms. 

Basically,  this  paper  presented  two  proposi- 
tions. First,  that  computer  system  design  eval- 
uation activities  should  follow  the  six  steps 
outlined  in  Part  II,  and  that  these  activities 
should  be  conducted  by  a design  evaluation 
team  made  up  of  users,  design  specialists,  and 
development  specialists.  Early  use  of  this  pro- 
cedure introduces  discipline  to  the  process 
that  might  otherwise  be  missing.  Use  of  an 
integrated  team  of  users,  designers,  and  de- 
velopers, tends  to  minimize  the  communica- 
tion gap  problems  and  probably  shortens  the 
overall  process. 

The  second  proposition  is  that  computer 
system  simulation  is  a valuable  tool.  This  tool 
can  be  used  effectively  during  the  validation 
phase  to  articulate  designs  in  the  form  of 
readable  models,  and  to  gather  valuable  esti- 
mates of  performance  for  each  proposed  de- 
sign. 

A technique,  model,  or  procedure  cannot 
totally  eliminate  the  risks  associated  with  de- 
veloping complex  weapon  systems  containing 
embedded  computers.  The  procedures  out- 
lined and  illustrated  in  this  paper  are  pro- 
posed because  they  tend  to  provide  informa- 
tion about  the  opportunities  to  evaluate 
computer  system  designs  in  a logical  way,  and 
in  an  environment  where  hypotheses  can  be 
tested.  As  such,  computer  system  simulation 
and  the  six-step  procedure  in  which  it  is  em- 
ployed can  never  totally  eliminate  design 
risk,  but  can  aid  in  achieving  a significant  re- 
duction in  design  risk.  □ 


NOTE:  A paper  prepared  by  Major  Feingold  during  Program  Management  Course  PMC 
76-1,  Defense  Systems  Management  College,  formed  the  basis  for  this  article.  The  paper 
bears  AD  No.  A026387  and  is  available  from  the  Defense  Documentation  Center  upon 
request. 
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CORRECTIONS  Vol.  l,No.3-  SUMMER,  1977 


Page  ix. 

Line 

6, 

Reads:  “Major  John  G.  Albert,  US  Air  Force,  Commandant' 
Should  read:  Major  General  John  G.  Albert,  US  Air  Force, 
Commandant 

Page  L 

Line 

1. 

Headline  reads:  “The  Process  of  Standarization” 
Should  read:  The  Process  of  Standardization 

Page  43. 

Line 

1. 

Headline  reads:  “NATO  STANDARIZATION” 
Should  read:  NATO  STANDARDIZATION 

Page  65. 

Line  12. 

Reads:  “Ms  Avondale  L.  Stephenson” 
Should  read:  Ms  Avonale  L.  Stephenson 
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